Comparing Microsoft Team Foundation Server and Collection of Open Source Tools

Over the past few months I've been formalizing my deployment process through the use of configuration management tools.  I have what I know feel is a nice environment for doing continuous integration builds, a file is checked the file in, and out pops a build.  Using branching in the TFS Source Control allows for promoting changes from a development to staging as this happens the continuous integration environment picks up that change to the staging branch and deploys the new build to our User Acceptance environment.

Now what I want to do is associate work items completed with the actual builds, since Software Logistics has minimal support staff, this needs to be automated as much as possible.  Basically the flow of a work item should be as follows:

1. End User/Manager enters a bug/task, for the sake of this work flow, let's say this automatically gets assigned to the person responsible for closing the work item.
2. The person responsible for closing the work item should be notified via email
3. At some point we need to figure out scheduling
4. The developer works on the work item
5. Checks in the source code
6. This kicks off a build
7. An RSS Item is created by the build process, included within this RSS Item is the Work Items completed since the last build.
8. Once the manager decides it's time to deploy the changes, they promote the changes to the staging or user acceptance environment
9. This kicks off an additional build, we know the last time the build took place so now we want to associate work items with this build.  This should also be in the form of an RSS entry the client can subscribe to.
10. Included in this RSS entry is the build that can be applied to the production enviornment. 

For now, I've pretty much settled on using the Version Control software that is built in to TFS and Visual Studio 2005 for my IDE, everything else is up in the air.  In addition, this environment should be peiced together with as little new custom projects as possible. 

The two alternatives for putting this environment together are:

1. Fully moving to TFS to manage all of this
2. Peice together a build environment that leverages open source tools.

Microsoft Team Foundation Server Environment

What is really nice about this is that it is an all encompassing solution. 

The following features are built in:

• Work Item Tracking (very customizable)
• Source Control
• MS Build Server

Over the last couple of days, I completed a first version of a Continous Integration environment.  If you are not familar with what Continous Integration is, I would recommend an excellent podcast from Scott Hanelman.

I have a dedicated build machine setup that will be hosting two environments for performing build & tests, I'm going to test

• Built In build server from Team Foundation Server
• CruiseControl.NET a product from ThoughtWorks

I had installed the Build Component with TFS a while ago, configured it to run builds and this seemed pretty cool, but haven't spent too much time on this.

The CruiseControl.NET environment I created is fairly basic, but I was able to get it up and running within about an hour.  Here is a summary of what is currently in place.

• CruiseControl.NET runs as a service and is configured with ccnet.config file.
• Within ccnet.config, I created a project that looks at source code within my TFS repository.
• When I check a file in to the TFS repository for the project I created, it get's all the source code under the node specified in the ccnet.config, then runs a task to kick off a build with MSBuild.
• A small application can be installed on our development machine that runs in the tray that allows you to check the status of builds, it also gives you notifications of when builds are completed or they break.
• In addition there is a web based dashboard that will give me a report of the latest builds.

I'm extremely excitied about using additional features that will allow for unit testing and functional testing as well as deployment.  I'm going to compare the Microsoft TFS offerring with the CruiseControl option.  In theory, if I build these both around the the MSBuild tool, I should be able to have similar build processes in both environments.

What experiences have you had with continuous integration or similar technologies?

Upgrading from Beta 3 refresh of TFS to RTM, was as painful as I expected (but probably my own fault).  With great hopes, I downloaded the Team Foundation Server Upgrade Package and faithfully followed the steps within the document. 

The first thing I learned when applying the update was the Beta 3 Refresh of TFS was incorrectly installed.  I purchased a server off of ebay to have a dedicated machine running this software, however when I installed SharePoint services, I missed the step to install it with the "Server Farm" option.  This created a named instances of SQL Server independent of the SQL install hosting TFS.  When the TFSUpgrade utility attempted to apply the updates to the SharePoint databases, it failed since the upgrade was looking in the default instances.  I moved the databases from the SharePoint instance to the default instance, and I think that was the cause of most of the problems.

To make a long painful story short here are the next steps I did to complete the effort...

• Ensure I had backups of all the databases involved (before and after I ran the TFS Upgrade Utility).
• Uninstall Beta3 refresh of TFS
• Attempted to install the RTM version, got SharePoint errors (stmadm.exe did not return zero)
• Attempted new settings in SharePoint for "Default Config" db, as well as the "Content DBs" for virtual servers.
• Uninstalled SharePoint, reinstalled with Server Farm option selected (this allows for TFS to manage the database configuration).
• Attempted to install RTM version again, got similar SharePoint errors.
• After trying different combinations, looking at the data within the STS databases, after about 4 hours I decided to just rebuild the box install TFS from scratch.
• This went well, everything worked as expected and within about 1.5 hours I went from a formatted hard-drive to an up and running TFS server (probably would have been quicker, but this is an older PIII Xeon @ 1.1GHz)
• For the moment of truth...I restored the backups of all my databases...DOH, when I rebuilt the box, I had thought hmmm, it was named SLNETPSQL3 before, let's name it SLNETTFS this time...big mistake!!!  After I restored, all the data referenced the name of the original server SLNETPSQL3, I probably spent about an hour or two attempting to figure out how to change everything over, but no luck...
• Decided to just go ahead and rebuild it one more time, this time with the original server name.  Again about 1.5 hours later I had a fresh install of TFS with everything working just fine.  I made a copy fo the original DB's.
• Next I attempted to restore the original DB's, at soon as I did this, I couldn't connect to TFS with any of the users.  So at at this point, I had about 20 hours into the upgrade and needed to get some work done to pay the bills, so I decided to just restore the databases from the fresh copy that I knew worked.
• I was in business, then I just restored the versions of my production TFSIntegration db to a different name, copied the tables tbl_projects, tbl_security_projects & tbl_security_objects for those projects into my new database.
• At that point, my projects came on line.
• I restored the TFSVersionControl database and all my source history got restored YIPPPPPEEEEE!!! (I had off-line backups of the source, but it was nice to be able to keep the source history)
• Next I had thought, ok let's get the work items in there...no luck here, there is data in the TFSIntegration database that needed to be tied to the data within the work items db, so this did not restore to work with my new version of the projects in the TFSIntegration DB.
• I tried for about a couple of hours to manually import the tables, but ran in to some F-Key violations that stemmed form the work items not being created for the projects (I think I know what I need to do to restore these, but again need to get some real work done to pay the bills, probably do another post when I get this completed.

Anyway - things are all well, I have a couple of legacy TFS projects that are limping along but useable non-the-less.

Has any one else had similar experiences?

*Note - Although my experience documented here wasn't too "pleasant" it is probably due to my lack of my prior knowledge on TFS configuration and architecture and learning on the fly.  This was an excellent learning experience and the more I learn about the capabilities of this product, the more I like it!