TFS 2010 Beta2: TF255144: Exception has been thrown by the target of an invocation

Symptoms

Do you have an error during installation that looks something like this:

Does your log file contain an error that like this:

[Info   @08:19:12.404] Starting website 8080
[Error  @08:19:12.705] PRODUCT ISSUE: UNHANDLED EXCEPTION
[Error  @08:19:12.717] System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> System.Runtime.InteropServices.COMException: The object identifier does not represent a valid object. (Exception from HRESULT: 0x800710D8)
   --- End of inner exception stack trace ---
   at System.DirectoryServices.DirectoryEntry.Invoke(String methodName, Object[] args)
   at Microsoft.TeamFoundation.Admin.IISHandler.StartWebSite(OperationContext context, Int32 websiteId)
   at Microsoft.TeamFoundation.Admin.IISWebSite.Start(OperationContext context)
   at Microsoft.TeamFoundation.Admin.ATInstall.PostWebInstallStuff(OperationContext context)
   at Microsoft.TeamFoundation.Admin.ATInstall.PostWebUpdateStuff(OperationContext context)
   at Microsoft.TeamFoundation.Admin.ApplicationTier.Install(OperationContext context)
   at Microsoft.TeamFoundation.Admin.ApplicationTier.Apply(OperationContext context)
   at Microsoft.TeamFoundation.Admin.ConfigurationNode.ApplyIfReady(OperationContext context)
   at Microsoft.TeamFoundation.Admin.LogicalTier.Apply(OperationalMode mode)
[Error  @08:19:12.721] TF255184: An error occurred during operation. Message=TF255144: The following error occurred during Apply: Exception has been thrown by the target of an invocation...

I have highlighted the critical part of the callstack.

The Good News

We have fixed this bug (post Beta2).  We first hit this bug in early October and it was fixed on the 8th.  We’ve had one user report hitting this “in the wild” – a fraction of 1% of the overall installations.  So it’s not pervasive and it’s not destructive – two nice qualities in a bug.

Good for you.  How about me?

If you hit this bug while installing Beta2 my best advice is just to try again.  Melborp was kind enough to share workaround steps in his Connect report.  Start over and it will likely work the second time.  If it happens repeatedly let me know and we can see how to work passed it.  So far this has not become a blocking bug for anyone.

More Info

http://connect.microsoft.com/VisualStudio/feedback/ViewFeedback.aspx?FeedbackID=504392  (same link as above)

Enabling Tracing during TFS 2010 Configuration

Please note: this post applies only to Team Foundation Server 2010 (Beta 1 to current).  It does not apply to TFS 2005, 2008 or any future version (though I suspect that any future version would have similar capabilities).

Were you asked to enable tracing during TFS configuration?  Is the configuration process crashing and not leaving around a sensible log (What? That can’t happen!).  Are you just really curious about what is happening during configuration and want to see all the gory details?

I’ll repeat this several times in this post – the tracing we are enabling is for the Application Tier configuration tools only.  These steps take place on your Application Tier but in NO WAY do they affect the TFS services performance or behaviors – this includes the web services, build services, job agent service and anything else that is not tfsconfig.exe or tfsmgmt.exe (i.e. the command line and GUI based TFS 2010 configuration tools).

If so – this is the post you’re looking for.

Enabling Tracing

1. Close the TFS 2010 configuration wizard or admin console (if running).
1. We only check the trace level when the program starts up.  So if it’s already running and you haven’t done this – you need to close it!
2. Oh – if you are running the TFS 2010 Setup and are at the window where you are asked if you want to launch the configuration tool – you can update the registry key now and not need to close the configuration wizard since the MSI and configuration phases are separate executables).
2. Enable verbose tracing by doing the following:
1. Open Regedit (must be an Administrator)
2. Find the key HKLM\SOFTWARE\Microsoft\TeamFoundationServer\10.0\Admin
3. Edit the value “TraceLevel” (DWORD) – set to “4” (it is probably “0”)
4. You can do this from an admin command prompt by running:
1. reg.exe add HKLM\SOFTWARE\Microsoft\TeamFoundationServer\10.0\Admin /t REG_DWORD /v TraceLevel /d 4 /f
3. Run DbgView
1. You can download DbgView from http://technet.microsoft.com/en-us/sysinternals/bb896647.aspx
1. You can also run it right from the website at this link: http://live.sysinternals.com/Dbgview.exe
4. By default DbgView is ready to go – Under the “Capture” menu make sure that “Capture Win32” (CTRL+W), “Pass-Through” and “Capture Events” (CTRL+E) are enabled.  Nothing else is needed.
1. 
5. Start up the TFS Administration Console (or configuration tool, whatever).

When the config tool starts you should see some data streaming in to DbgView – it should look something like this:

[6468] Admin TraceLevel = Verbose

Hooray!  Tracing is enabled!!!!

Show Me The Errors!

1. Bring up the filters screen in DbgView (CTRL+L or clicking on this button: ). 
2. Leave the Include and Exclude fields alone.
3. In the Highlight section leave “Filter 1” selected and type “[Error” into the Red text area (please note that I typed an opened brace but not a closing one and also do not type the double quotes).  It should look like this:
1. 
4. Now change “Filter 1” to “Filter 3” (Nothing magically about 3.  I use 3 because it’s kind of orange – I don’t like the purple of “Filter 2” and I don’t want to get into color changing in this blog post – you want to change colors?  Go nuts.).
5. Add the string “[Warning” – again, no closing brace and don’t use the double quotes.
1. 
6. Now if you happen to have an error it will be much easier to detect.

Ok … so why am I doing this?

Let me give you an example of why.  Let’s say in the Advanced Configuration Wizard you type in the wrong name of the SQL Server Instance you want to use.  The error message you will get is:

But look at some of the trace output:

[6468] [Verbose@18:45:56.241] Checking if database master exists on SQL connection Data Source=HORVICKVM-DEV2;Integrated Security=True
[6468] [Error  @18:46:21.381] Microsoft.TeamFoundation.Admin.TfsAdminException: TF255049: A connection cannot be made to the server that is running SQL Server. Verify that you have entered the correct name for the server, including the instance name, that the server you are attempting to connect to is online, and that you have the required permissions to connect. ---> Microsoft.TeamFoundation.Framework.Server.DatabaseConnectionException: TF246017: Team Foundation Server could not connect to the database. Verify that the server that is hosting the database is operational, and that network problems are not blocking communication with the server. ---> System.Data.SqlClient.SqlException: A network-related or instance-specific error occurred while establishing a connection to SQL Server. The server was not found or was not accessible. Verify that the instance name is correct and that SQL Server is configured to allow remote connections. (provider: Named Pipes Provider, error: 40 - Could not open a connection to SQL Server)

What’s so great?

Well first the error message is highlighted red (in DbgView it will be white text on a red background given my example above – but here I made it red text on white to be more readable in the blog post).

Also the message contains the true .NET Exception that occurred and the connection string we used.  Those are two really useful pieces of information when you are trying to figure out what is happening.  More specifically it even told you how the check was being performed (“Checking if database master exists…”).

Oh No!  I have Warnings and Errors!

Well – did anything actually go wrong with your configuration?  If not – then it’s probably nothing to worry about.  We have some messages that come out as warnings or errors but are in fact benign.

Why Don’t I See Server Messages?

This blog post only explains how to enable tracing for the TFS 2010 admin tools.  It does not explain how to enable tracing for the server components, SQL, the job service (though we do trace the job service output when a servicing operation is initiated by an admin tool such as tfsmgmt.exe or tfsconfig.exe).  This trace data will provide you zero insight into your server’s realtime activities.  Only configuration actions.

Should I Leave It This Way?

No, probably not.  It’s meant as a debugging aid.

We do perform tests with tracing enabled (it is constantly enabled on all my machines) – but it’s not the “pure” way to keep your system. 

That said…

Tracing in no way affects your server performance.  I repeat (again and again) – enabling tracing in no way affects your TFS server performance.  Nor does it affect the operational behavior of any TFS feature or service (other than itself, obviously).

It could negligibly impact the configuration tool performance.  But do you think tracing out a string is more or less expensive than creating a connection to a remote SQL server and executing tens of SQL scripts?  It’s not even close.  Tracing is orders of magnitude cheaper than most of our configuration actions.

I personally keep DbgView open 24/7 on my machines and any time something goes wrong I pop it open and see what happened.  That’s how I roll.  I’m wild like that.

Reading Twitter Data with C# and LINQ

I wanted to read Twitter.com search results (tweets) using C#.  I started by deciding that a tweet looks something like this:

    public class Tweet
    {
        public string Id { get; set; }
        public DateTime Published { get; set; }
        public string Link { get; set; }
        public string Title { get; set; }
        public Author Author { get; set; }
    }
    public class Author
    {
        public string Name { get; set; }
        public string Uri { get; set; }
    }

Next I needed a way to get the data from the web and into a collection of Tweet’s.  I defined a TweetStream type that would handle the following:

1. Downloading the data from Twitter.com
2. Deserializing the Tweet data
3. Remembering the high-watermark so that we always only get new tweets.

I stubbed out this:

    public class TweetStream
    {
        private string m_refreshUri;
        List<Tweet> m_tweets;
        public TweetStream(string queryUri)
        {
            m_refreshUri = queryUri;
            m_tweets = new List<Tweet>();
        }
        public List<Tweet> Tweets
        {
            get
            {
                return m_tweets;
            }
        }
        public void Refresh()
        {
            // TODO - download tweet information from Twitter.com,
            // populate the tweet collection with the new
            // tweet data, and store the new high-watermark
        }
    }

Now I just need to fill in Refresh.

Download the results from Twitter

The LINQ XDocument type makes this very easy.  What’s not clear here is that I am loading ATOM data using the Twitter ATOM search API.  The result stream will look something like this:

<?xml version="1.0" encoding="UTF-8"?>
<feed xmlns:google="http://base.google.com/ns/1.0" xml:lang="en-US" xmlns:openSearch="http://a9.com/-/spec/opensearch/1.1/" xmlns="http://www.w3.org/2005/Atom" xmlns:twitter="http://api.twitter.com/">
  <id>tag:search.twitter.com,2005:search/TFS</id>
  <link type="text/html" rel="alternate" href="http://search.twitter.com/search?q=TFS"/>
  <link type="application/atom+xml" rel="self" href="http://search.twitter.com/search.atom?q=TFS"/>
  <title>TFS - Twitter Search</title>
  <link type="application/opensearchdescription+xml" rel="search" href="http://search.twitter.com/opensearch.xml"/>
  <link type="application/atom+xml" rel="refresh" href="http://search.twitter.com/search.atom?q=TFS&amp;since_id=1152416895"/>
  <twitter:warning>adjusted since_id, it was older than allowed</twitter:warning>
  <updated>2009-01-27T16:13:14Z</updated>
  <openSearch:itemsPerPage>15</openSearch:itemsPerPage>
  <openSearch:language>en</openSearch:language>
  <link type="application/atom+xml" rel="next" href="http://search.twitter.com/search.atom?max_id=1152416895&amp;page=2&amp;q=TFS"/>
  <entry>
    <id>tag:search.twitter.com,2005:1152416895</id>
    <published>2009-01-27T16:13:14Z</published>
    <link type="text/html" rel="alternate" href="http://twitter.com/bubbafat/statuses/1152416895"/>
    <title>All day TFS training.</title>
    <content type="html">All day &lt;b&gt;TFS&lt;/b&gt; training.</content>
    <updated>2009-01-27T16:13:14Z</updated>
    <link type="image/png" rel="image" href="http://static.twitter.com/images/default_profile_normal.png"/>
    <author>
      <name>bubbafat (Robert Horvick)</name>
      <uri>http://twitter.com/bubbafat</uri>
    </author>
  </entry>
  <entry>...</entry>
  <entry>...</entry>
  <entry>...</entry>
</feed>
And we load it like this: 

XDocument feed = XDocument.Load(m_refreshUri);

Populate the Tweet Collection

With the document loaded we can now start pulling out the data we want.  First we need to define the ATOM namespace:

XNamespace atomNS = "http://www.w3.org/2005/Atom";

Next we need to iterate over every “entry” element in the feed and extract the data we care about.  LINQ makes this a breeze (though perhaps a bit tough to debug and grok at first).

m_tweets = (from tweet in feed.Descendants(atomNS + "entry")
    select new Tweet
    {
        Title = (string)tweet.Element(atomNS + "title"),
        Published = DateTime.Parse((string)tweet.Element(atomNS + "published")),
        Id = (string)tweet.Element(atomNS + "id"),
        Link = tweet.Elements(atomNS + "link")
            .Where(link => (string)link.Attribute("rel") == "alternate")
            .Select(link => (string)link.Attribute("href"))
            .First(),
        Author = (from author in tweet.Descendants(atomNS + "author")
            select new Author
            {
                Name = (string)author.Element(atomNS + "name"),
                Uri = (string)author.Element(atomNS + "uri"),
            }).First(),
    }).ToList<Tweet>();

Store the High-watermark

Finally we want to load the high watermark so we don’t ask for data twice.  This is just a matter of loading the “refresh” link from the top-level feed elements.

m_refreshUri = feed.Descendants(atomNS + "link")
    .Where(link => link.Attribute("rel").Value == "refresh")
    .Select(link => link.Attribute("href").Value)
    .First();

Putting it all Together

Not to put it all together you can create a collection of TweetStream’s and refresh them in a loop – printing out the tweets as you find them.

using System;
using System.Threading;
using TwitterLib;
namespace TestHost
{
    class Program
    {
        static void Main(string[] args)
        {
            int totalTweets = 0;
            TweetStream[] tweetStreams = new TweetStream[] {
                new TweetStream("http://search.twitter.com/search.atom?q=TFS"),
                new TweetStream("http://search.twitter.com/search.atom?q=Team+Foundation+Server"),
                new TweetStream("http://search.twitter.com/search.atom?q=TFS2008"),
                new TweetStream("http://search.twitter.com/search.atom?q=TFS2005"),
            };
            while (true)
            {
                int currentTweets = 0;
                bool newTweets = false;
                foreach (TweetStream stream in tweetStreams)
                {
                    stream.Refresh();
                    foreach(Tweet tweet in stream.Tweets)
                    {
                        Console.WriteLine("{0}: {1}",
                            tweet.Author.TwitterId,
                            tweet.Title);
                        newTweets = true;
                        totalTweets++;
                        currentTweets++;
                    }
                }
                if (newTweets)
                {
                    Console.WriteLine("Loaded {0} more tweets", currentTweets);
                    Console.WriteLine("Loaded {0} total tweets", totalTweets);
                }
                else
                {
                    Console.WriteLine("No new tweets.");
                }
                DateTime nextCheck = DateTime.Now.AddMinutes(5);
                Console.WriteLine("Will check again at {0}", nextCheck.ToShortTimeString());
                Thread.Sleep(TimeSpan.FromMinutes(5));
            }
        }
    }
}

You may have noticed I used a property (Author.TwitterId) that is not in the code – it’s just a simple Regex:

public string TwitterId
{
    get
    {
        return s_idParser.Match(Name).Groups["twitterid"].Value;
    }
}
private static Regex s_idParser = 
    new Regex(@"^(?<twitterid>.*)\s+\((?<displayname>.*)\)");

Whats’ Next?

To be clear this is not production-ready code.  It is very optimistic, quite inefficient and a bit ugly.  Also it doesn’t do anything terribly interesting.

But in less than 100 lines you have an ugly, slow, optimistic Twitter feed reader (which could read any ATOM stream with just a few additional lines).

How I Use Hyper-V to Configure my Development Environment

Prior to using Hyper-V as part of my daily development and test efforts I wasted man weeks every year.  I lost days at a time configuring machines and recovering from updated NDP builds.  I wasted hours every week repeating the same steps over and over.

I will be bold enough to say that Hyper-V allows me to get at least 15% more work done each week on less than a third of the hardware I had before and I never have long downtimes due to my machine being down.  Even a total machine failure is not going to put me offline for longer than it takes to reinstall Windows and Office.

So how did I setup my environment?

Hardware

• Dell OptiPlex 755
• Intel Core 2 Quad (Q6600) @ 2.4 Ghz
• 8 GB RAM
• 250 GB HD (Primary)
• C: – 25GB (boot drive)
• F: – 225GB (data drive)
• 500 GB HD (Secondary)
• E: (data drive)
• 150 GB HD
• Marked Offline in Windows (you can do this through disk management – it must be a real drive, not just a logical partition)
• 500 GB Western Digital MyBook HD (external USB drive)
• Monitors
• Samsung SyncMaster 206BW (20” LCD)
• Philips 170B (17” LCD)
• MidiLand Speakers (el-cheapo junk speakers I found in a box somewhere)

The overall hardware investment is about $1300 (and that’s retail pricing that anyone can get).

To create my development environment I do the following (applying the latest patches at every step and CA eTrust is installed on every machine [physical and virtual])

Configure Primary Box

1. Install Windows 2008 (64bit) on the C: drive
2. Install Office 2007 on the C: drive
3. Install the Hyper-V services (RTM)
4. Create the folder F:\VM
5. Plug my Zune directly into the speakers – I don’t want to give up CPU/RAM for music to play.
6. Configure Windows Backup to backup my C: and F: drive to the external hard-drive (that’s why it does not have a drive letter – Windows takes over the drive).  I do nightly backups of my VMs.

Configure Development Box

My goal with the dev box is to optimize for quick resets when we take new builds of Visual Studio or the CLR.  Since I work on the VS team I take new builds every few weeks.  To do this I pre-install a lot of stuff, create some baseline snapshots and then mount the physical disk for my source enlistment.  Mounting physical disk is REALLY IMPORTANT.  You do NOT want your source code on a virtual drive because when you revert to an older snapshot you will lose it’s contents and that will both screw up your workspace contents (requiring a force sync) and may result in losing data (files that were added/edited but not checked in).  Also the physical drive means build performance is not impacted by a virtualized drive.  The virtual drives in Hyper-V (even the expandable ones) have great performance – but moving the build to a physical drive really helps keep things rolling.

One caveat – you cannot take snapshots with a physical drive mounted so you do not get live snapshots and you must unmount whenever you apply a previous snapshot (this is trivial, but for some reason the Apply operation won’t do it for you behind the scenes).

I can recover from an VS/NDP update in about 2 hours with this model.  On physical hardware it would mean formatting and starting over – costing about a day.

1. Create a new (127GB - Expandable) virtual disk at F:\VM\Disks
2. Create a new virtual machine (at F:\VM\Machines) called “HORVICKVM-DEV” with
1. Four Virtual Processors
2. 2 GB of RAM
3. Legacy Network Adaptor (So I can use RIS for network installs)Boot to RIS network boot
4. The drive you created in step #1
3. Install Windows 2003 (32 bit) [Choose whatever OS you want]
4. When it is finished installing shut the machine down and take a snapshot called “OS Installation Complete (OFF)”
1. I always put “OFF” or “LIVE” in my snapshot names so it’s clear to me what the running state of the snapshot is without having to look at the details or preview window.
5. In the VM configuration settings for the dev box
1. Change the network adaptor from Legacy Network Adaptor to “Network Adaptor”
6. Boot the Development VM
7. Install NDP 2.0, and 3.5 + SP1
8. Install Source Insight
1. I always want to have an editor on the box in the event I run into a VS bug I can’t work around.
9. Install WinDbg and configure it for use with our symbol and source server plus my favorite configuration tweaks.
1. I always want to have a debugger installed in case I need to debug the VS install (and frankly WinDbg + SoS is pretty damn powerful).
10. Shut Down the VM
11. Snapshot named “Editor and NDP 3.5+ Patched (OFF)”
1. Now – Once I’m here I never need to do any of the above steps again.  Ever.  This is my persistent baseline snapshot that I will love forever.  In the future I will come back to this snapshot, apply patches and make a new baseline (deleting this snapshot and renaming the new one to this name)
12. Boot the VM
13. Install the build of Visual Studio we are using at the moment (this includes NDP4, etc)
14. Start VS and setup my configuration settings (C# environment, debugger and symbol options, editor options, etc)
15. Shut down the VM
16. Snapshot the VM as “Installed VSTS and NDP4 build XXXXX (OFF)”
1. When we take a new version of VS I will need to abandon this snapshot tree so it was important that I did all the OS and non-CLR/VS related changes prior to this snapshot.
17. In the Hyper-V MMC bring up the settings window for this VM
1. Select “IDE Controller 1”
2. Select “Hard Drive” and click the Add button
3. Choose the “Physical hard disk” radio button and select the drive marked offline in the drop down list (it should be your only option since only offline drives will show up here).
4. Click OK to save the drive in the settings.
18. Boot the VM
19. On the new drive I create a folder named “HORVICKVM-DEV” (the machine name) and under it enlist in the source code for my branch.
1. I create the folder with the machine name because I sometimes mount the same drive under multiple VMs (not at the same time) and this helps me isolate changes.
20. Build, Test – Ready!

When I take a new VS I will revert to step 12 and start over (I do export my configuration settings to make configuring VS easier).

Configure Test Box 1 (2 and 3)

My goal with the test boxes is, like the dev box, to be able to quickly recover from NDP updates but also to have a stable SQL and IIS environment to start from.  I will create multiple identical test boxes that differ only in OS (2003 or 2008) and data width (32 or 64).

I can recover from an NDP update in about 30 minutes with this model.  On physical hardware this would also be about a day.

1. Create a new (127GB - Expandable) virtual disk at F:\VM\Disks
2. Create a new virtual machine (at F:\VM\Machines) called “HORVICKVM-TEST1” with
1. Two Virtual Processors
2. 1 GB of RAM
3. Legacy Network Adaptor (So I can use RIS for network installs)Boot to RIS network boot
4. The drive you created in step #1
3. Install Windows 2003 (32 bit), Windows 2008 (32bit) or Windows 2008 (64 bit)
1. Since I work on the server component I don’t install a client OS on my test boxes
2. When it is finished installing shut the machine down and take a snapshot called “OS Installation Complete (OFF)”
4. In the VM configuration settings for test box 1
1. Change the network adaptor from Legacy Network Adaptor to “Network Adaptor”
5. Boot the Test VM
1. Install NDP 2.0, and 3.5 + SP1
2. Install WinDbg and configure it for use with our symbol and source server plus my favorite configuration tweaks.
6. Shut Down the VM
7. Snapshot named “NDP 3.5+ Patched (OFF)”
8. Boot the VM
9. Install SQL server 2008
10. Install IIS 6 or 7 (depending on OS)
11. Copy my test scripts from my persistent network share to a well known location on the virtual drive (I always want these in the baseline – they will update themselves if newer versions exist after a snapshot revert)
12. Shut down the VM
13. Snapshot named “SQL 2008 + IIS (OFF)”
1. Now – Once I’m here I never need to do any of the above steps again.  Ever.  This is my persistent baseline snapshot that I will love forever.  In the future I will come back to this snapshot, apply patches and make a new baseline (deleting this snapshot and renaming the new one to this name)
14. Boot the VM
15. Snapshot the VM as “Before NDP4 (LIVE)”
1. In case the NDP4 install has problems I want a live clean snapshot I can quickly get back to
16. Install the version of the NDP4 that matches the version installed on my dev box
17. Snapshot the VM as “NDP4 XXXXX installed (LIVE)”
18. Done!

The machine is now ready for testing.  Did you notice that WSS was not installed?  If I ever want it to be persistently installed I will create a new branch off the snapshot “NDP4 XXXXX installed (LIVE)” with WSS installed.  Typically I will either install TFS choosing to let it install WSS or I will have it not configure WSS.

Fun Things I Can Do Now …

• After test runs complete I can revert back to the snapshot “NDP4 XXXXX installed (LIVE)” and start over (or just re-run them if they are non-destructive).  When I take patches I start with the “NDP4 XXXXX installed (LIVE)” snapshot, apply them there and then make that the new baseline.
• Say I want to test the MSI out with four or five different configurations.  Say it takes 8 minutes to get the MSI to the point where I want to test.  I can save over 30 minutes by doing a live snapshot of the test machine at the point where I want to branch the testing and then after each attempt re-apply that live snapshot.  As long as I don’t muck with network resources (e.g. external SQL or WSS servers) this works great.
• Hardware/OS configurations.  On a single box I can test several OS and data width options.  This saves the company thousands of dollars and me hours of time.  Better still my code is better tested and that saves our customers countless hours of frustration and pain.
• Limited memory testing.  I can test deploying TFS on a single tier (SQL + WSS + RS + TFS) with, say, 512 MB of RAM (or less).  I don’t need to pull sticks or disable things in the BIOS.  This saves me hours and lets me cover scenarios I wouldn’t otherwise.
• Single/Multi-Proc configurations.  I have a quad core machine but say I want to test a bug that only repros on single core machines – well – I can!  Or 2.  Or 4.  It’s a nice balance.
• Using 1GB footprints I can run several test VMs at once to test multi-tier configurations (SQL on one box, RS on another, WSS on another, TFS on another) that before would have been too difficult to coordinate easily and nearly impossible to repeat numerous times per hour (at my desk – our test team has tools that can help them do this in a lab environment).

Yeah – I know a lot of you are reading this going “Well … DUH!  I’ve been using VMWare/Xen/Whatever to do that for years …”  I’m not saying this is new.  What I’m saying is that for me, as someone who didn’t have the option to use those before, this is a game-changing way of development.

I will never go back to piles of boxes under my desk.

Credit is due to James Manning for pointing out several of the configuration choices I’ve described above.

Windows 7 Feature I Like – Programmer’s Calculator

I just installed Windows 7 beta on a Hyper-V VM (1GB RAM, single proc, 127 expandable drive) and found that the calculator has changed and I like it.

Beyond Standard and Scientific there are two new modes, Programmer and Statistics.

Programmer supports Binary, Hex, Octal and Decimal and includes a slick variable length (byte to qword) binary view.

Since I’m currently working with quite a lot of bit twiddling this is a handy mode that I will be using frequently.

Statistics mode is far less interesting and has features to do things I no longer recall the usefulness of.  It’s all very important, I’m sure.

New VSTS and TFS Virtual PC/Server and Hyper-V images are available!

Brian Randell has been hard at work putting together a holiday surprise we all will enjoy.  New Virtual PC/Server and Hyper-V images with VSTS and TFS and a 12/31/2009 expiration date.

Please see his blog post for the full details - I will re-post a little here.

 http://www.pluralsight.com/community/blogs/brian/archive/2008/12/24/happy-holidays-and-look-what-santa-s-brought.aspx

  --

As part of the refresh and extending the expiration date, I’ve updated the images to include the following (naturally some items only appear in the “all-up” images):

• the latest virtual machine additions or integration services components
• all of the latest Windows Updates as of December 1, 2008
• Team Foundation Server 2008 SP1
• Visual Studio 2008 Team Suite SP1
• Team System Web Access 2008 SP1
• Team Foundation Power Tools, October 2008 update
• the latest MSSCCI provider
• Team Explorer 2005 (this allows you to work with TFS from Visual Studio 2005 which is installed so that you can edit and customize TFS reports)
• the GDR for the product formally known as Visual Studio 2008 Database Edition (aka Data Dude)
• current Process Explorer, Process Monitor, and Background Info

Naturally at this point, you’re wondering, where are the goods? Here they are:

• VSTS “all-up” Virtual PC/Virtual Server image (6 GB download, expands to 15 GB)
• TFS “only” Virtual PC/Virtual Server image (3 GB download, expands to 8 GB)
• VSTS “all-up” Hyper-V image (6 GB download, expands to 15 GB)
• TFS “only” Hyper-V image (3 GB download, expands to 8 GB)

The Virtual PC 2007/Virtual Server 2005 R2 compatible images provide VMC files for Virtual PC 2007 SP1 and Virtual Server 2005 R2 SP1 Update. The VMC file that ends in –V7 is for Virtual PC and the one that ends in –R2 is for, well, Virtual Server.

The Hyper-V downloads only contain the VHD. You’ll need to configure them yourself. I wanted to make it as easy as possible.

TFS Branching Guidance Version 2 has been Released

 The VSTS Ranger team has released TFS Branching Guidance II.  This document describes pragmatic branching practices that are based on the real world experience of TFS professionals inside and outside of Microsoft.

What's new?  I'l just repeat their information:

• TFS Branching Guide - Main 2.0
• This is the main article which briefly explains branching concepts and introduces 3 levels of the most common branching scenario
• TFS Branching Guide - Scenarios 2.0
• A collection of less common branching scenarios
• TFS Branching Guide - Q&A 2.0
• A set of most frequently asked questions with answers
• TFS Branching Guide - Drawings 2.0
• A set of branching drawings in different formats including a large branching poster
• TFS Branching Guide - Labs 2.0
• A couple of examples for hands on labs with step by step instruction for practicing the branching scenarios

 Check it out!

 http://www.codeplex.com/TFSBranchingGuideII

Using a little Ruby to automate mstest code coverage runs

I hate repetitive tasks - especially ones that are easily automated.  Generating code coverage runs using mstest is an example of that.

The goal was to create a script that could do that following:

1)     Validate that the input binaries were available

2)     Validate that the necessary runtime components were available

3)     Instrument the input binaries

4)     Build up the test sandbox

5)     Start the coverage service

6)     Perform the tests using the instrumented binaries

7)     Stop the coverage service

Being able to start the process at any point in there was bonus points.

I choose to use ruby, and specifically Rake, to automate this process.  Rake is perfect for this type of task.  I can easily break the tasks down into their core parts and build up the automation one bit at a time.  Creating dependencies between the tasks is trivial.  It reads well (even with my poor ruby skills) and has a very nice command line experience.

What I ended up with was a process where I could do this:

C:\ruby\coverage>rake --tasks

(in C:/ruby/coverage)

coverage coverage:all       # Perform a complete code coverage pass...

coverage coverage:clean     # Clean the coverage temp directory and sandbox

coverage coverage:cover     # Instrument the assemblies for code coverage

coverage coverage:merge     # Merge the test results

coverage coverage:populate  # Populate the sandbox

coverage coverage:restore   # Restores the backed up files if they exist

coverage coverage:run       # Run the test suite

coverage coverage:start     # Start the coverage service

coverage coverage:stop      # Stop the coverage service

I can do all the things I set out to do (using “coverage:all”) or onesy-twosy as I need them (e.g. if rake aborts midway through the “run” step the coverage service will need to be manually stopped before the next run – I could modify a task to handle this more gracefully but I haven’t yet since the payoff isn’t there yet).

But you came for code ...

Rakefile

require 'rake'

require "win32/dir"

require 'fileutils'

require 'yaml'

def assert_config(config)

  ['vsperfcmd',

   'prodbin',

   'root',

   'sandbox',

   'covtemp',

   'covorig',

   'vsinstr',

   'suitebin']. each { |setting|

      assert_config_setting(config, setting)

  } 

end

def assert_config_setting(config, setting)

  fail "Setting \"#{setting}\" not found in coverage.yml" if config[setting].nil?

end

Rake.application.init('coverage')

config = YAML::load(File.open('coverage.yml'))

assert_config(config)

covfiles = FileList.new(

'Microsoft.TeamFoundation.Admin.dll',

'Microsoft.TeamFoundation.Management.Core.dll',

'Microsoft.TeamFoundation.Management.SnapIn.dll',

'Microsoft.TeamFoundation.Management.Controls.dll',

'tfsconfig.exe',

'tfsmgmt.exe'

)

namespace :coverage do

  desc "Instrument the assemblies for code coverage"

  task :cover => [:clean_coverage, :clean_orig ] do

    Dir.chdir(config['prodbin']) do

      mkdir config['covtemp']

      covfiles.each do |f|

        fail "Input file not found: #{f}" if not File.exists?(f)

        puts "Instrumenting #{f}"

        cmd = "\"#{config['vsinstr']}\" \"#{f}\" /COVERAGE"

        sh cmd

        assert_file_exists(File.join(config['covtemp'], f), cmd)

        copy_coverage(f, config['covtemp'])

      end

    end

  end

  desc "Restores the backed up files if they exist"

  task :restore do

    Dir.chdir(config['prodbin']) do

      covfiles.each do |f|

        origfile = "#{f}.orig"

        if File.exists?(origfile)

          rm f if File.exists?(f)

          mv origfile, f

          instrpdb = f.gsub(/(.*)\.(dll|exe)$/, '\1.instr.pdb')

          rm instrpdb if File.exists?(instrpdb)

        end

      end

    end

  end

  task :clean_backup do

    Dir.chdir(config['prodbin']) do

      covfiles.each do |f|

        origfile = "#{f}.orig"

        rm origfile if File.exists?(origfile)

        instrpdb = f.gsub(/(.*)\.(dll|exe)$/, '\1.instr.pdb')

        rm instrpdb if File.exists?(instrpdb)

      end

    end

  end

  task :clean_coverage do

    FileUtils.rm_rf(config['covtemp'])

  end

  task :clean_orig do

    FileUtils.rm_rf(config['covorig'])

  end

  task :clean_sandbox do

    FileUtils.rm_rf(config['sandbox'])

  end

  desc "Start the coverage service"

  task :start do

    mkdir config['sandbox'] unless File.exists?(config['sandbox'])

    sh "\"#{config['vsperfcmd']}\" /start:coverage \"/output:#{File.join(config['sandbox'], 'adminops.coverage')}\" /user:redmond\\vseqa1 /user:redmond\\tfssvc /user:#{ENV['USERDOMAIN']}\\#{ENV['USERNAME']} /waitstart"

  end

  desc "Stop the coverage service"

  task :stop do

    sh "#{config['vsperfcmd']} /shutdown"

  end

  desc "Run the test suite"

  task :run do

        Dir.chdir(config['sandbox']) do

          puts "Running Unit Tests"

          sh "aotest.exe /assembly:AdminOps.UnitTests.dll /test * /out:logs"

    end

  end

  desc "Merge the test results"

  task :merge do

  end

  desc "Populate the sandbox"

  task :populate=> [:clean_sandbox] do

    mkdir config['sandbox']

    puts "Copying Product binaries..."

    FileUtils.cp_r(Dir.glob(File.join(config['prodbin'], 'Microsoft.TeamFoundation.*')), config['sandbox'])

    cp File.join(config['prodbin'], 'NetFwTypeLib.dll'), config['sandbox']

    puts "Copying Test binaries..."

    FileUtils.cp_r(File.join(config['suitebin'], '.'), config['sandbox'])

    puts "Copying Instrumented binaries..."

    FileUtils.cp_r(Dir.glob(File.join(config['covtemp'], '.')), config['sandbox'])

  end

  desc "Perform a complete code coverage pass from scratch"

  task :all => [:clean, :cover, :populate, :start, :run, :stop, :merge]

  desc "Clean the coverage temp directory and sandbox"

  task :clean => [ :clean_coverage, :clean_sandbox, :restore, :clean_backup ]

end

def assert_file_exists(file, cmd)

  if not File.exists?(file)

    puts "COVERAGE ERROR: file not found: #{file}"

    puts "Executed: #{cmd}"

  end

end

def copy_coverage(file, dir) 

  if File.exists?(file)

    cp file, dir

  else

    die "Assembly not found for assembly #{file}"

  end

  pdbfile = file.gsub(/(.*)\.(dll|exe)$/, '\1.pdb')

  if File.exists?(pdbfile)

    cp pdbfile, dir

  else

    puts "WARNING: PDB file not found for assembly #{pdbfile}"

  end

  instrpdbfile = file.gsub(/(.*)\.(dll|exe)$/, '\1.instr.pdb')

  if File.exists?(instrpdbfile)

    cp instrpdbfile, dir

  else

    puts "WARNING: INSTR PDB file not found for assembly #{instrpdbfile}"

  end

end

coverage.yml

vsperfcmd: E:/Program Files/Microsoft Visual Studio 10.0/Team Tools/Performance Tools/vsperfcmd.exe

prodbin: C:/VMDEV/TfsArch1/binaries/x86chk/bin/i386

root: C:/VMDEV/TfsArch1/binaries/x86chk

sandbox: C:/SANDBOX

covtemp: C:/VMDEV/TfsArch1/binaries/x86chk/bin/i386/coverage

covorig: C:/VMDEV/TfsArch1/binaries/x86chk/bin/i386/coverage/orig

vsinstr: E:/Program Files/Microsoft Visual Studio 10.0/Team Tools/Performance Tools/vsinstr.exe

suitebin: C:/VMDEV/TfsArch1/binaries/x86chk/SuiteBin/i386/tfs/AdminOps/bin

So there you have it…

I go to my ruby command line (which is just a normal command line with the ruby bin directory in the path  - e.g. “set PATH=%PATH%;c:\ruby\bin”) and perform:

C:\ruby\coverage>rake coverage:all

When the tests are done the coverage file I created is in c:\sandbox ready to be imported into VS or merged with other files or dumped to excel or whatever.

And one note about the rakefile – you may notice that I call “aotest.exe” not “mstest.exe” – aotest is just a little wrapper around mstest that one of our test devs wrote.  I could have just as easily used mstest.

PDC Hours - Come and Talk Team Foundation Server Setup, Configuration and Administration

I'll be available to talk about TFS admin scenarios at PDC - my hours are:

Monday  - 12:30 - 6:00 in the VSTS Lounge

Tuesday  - 12:30 - 6:00 in the VSTS Lounge 

Wednesday  10:00-12:30 in the VSTS Lounge and Ask The Experts from 5:30-9:00PM

I won't be available Thursday so please find me before then!

Lets Talk TFS Setup, Configuration and Administration at PDC!

Coming to PDC?  Interested in Team Foundation Server Setup, Configuration and Administration?  Stop by the PDC Lounge and let's chat!

I'll post the hours I'll be available tomorrow.

A link to my Ruby and Rails blog

Since life can't be all Team Foundation Server and rainbows I've started getting serious about learning more about Ruby and the Rails framework.  To keep the TFS focus on this blog I've started a new blog dedicated to non-TFS tech topics http://www.GhostOnThird.com.

What to do when TfsQuiesce fails with "No associated service account for role TFSEXECROLE" error

When installing a service path on Team Foundation Server one of the first things that needs to happen is that TFS is "quiesced" - which basically means "to make quiet" - i.e. it's running but it's not accepting new client connections.  The tool that does this during the SP installation is called 'TfsQuiesce.exe". 

This blog post is to explain what you can do when you see the following in your log file after a failed upgrade:

TFSQuiesce - Team Foundation Server Maintenance Tool
Copyright (c) Microsoft Corporation.  All rights reserved.

Using workflow file from location exe.
Executing workflow 'Quiesce DT'...
Disabling SQL Jobs for databases TFSActivityLogging,TFSBuild,TFSIntegration,TFSVersionControl,TFSWorkItemTracking,TFSWorkItemTrackingAttachments,TFSWarehouse
Blocking service account from accessing database TFSActivityLogging
No associated service account for role TFSEXECROLE.
Executing workflow 'Unquiesce DT'...
Enabling SQL Jobs.
Unblocking service account from accessing database TFSActivityLogging
Unblocking service account from accessing database TFSBuild
Unblocking service account from accessing database TFSIntegration
Unblocking service account from accessing database TFSVersionControl
Unblocking service account from accessing database TFSWorkItemTracking
Unblocking service account from accessing database TFSWorkItemTrackingAttachments
Unblocking service account from accessing database TFSWarehouse

Workflow 'Quiesce DT' failed! ExitCode = 8000.
10/31/07 12:17:15 DDSet_Status: Process returned 8000
10/31/07 12:17:15 DDSet_Status: Found the matching error code  for return value '8000' and it is: '29206'
10/31/07 12:17:15 DDSet_Error:  8000
10/31/07 12:17:15 DDSet_CARetVal: 29206
10/31/07 12:17:15 DDSet_Status: QuietExec returned 29206

Notice the bolded line "No associated service account for role TFSEXECROLE" - this is the key line.

This error is indicating is that one or more of the databases have no members in the TFS exec role.  The reason we are running this check at all is to ensure that we are not running the upgrade as the service account user.  Let’s start by figuring out if this is a problem with all seven TFS databases or just one of them.

Run the following SQL statement 7 times – each time changing the “use TfsVersionControl” to one of the other TFS databases:

use TfsVersionControl

SELECT  dpMember.name
FROM    sys.database_principals dp
JOIN    sys.database_role_members drm
ON      drm.role_principal_id = dp.principal_id
JOIN    sys.database_principals dpMember
ON      dpMember.principal_id = drm.member_principal_id
WHERE   dpMember.Type = 'U'
        AND dp.name IN ('TFSEXECROLE')

I expect one or more of the result sets to be empty. 

Ultimately the role can’t be empty.  When you find the database(s) with the empty TFSEXECROLE role you should add the appropriate service account user to the role for that database(s).

Re-run the SQL snippet to ensure the fix was made properly and now running the installation should past this error.

Migration Toolkit Pre-release nears 100 downloads ...

At this time 98 people have downloaded the TFS Migration and Synchronization Toolkit.  Ok - some of you are saying "100?  What's the big deal?  A picture of a rabbit with a pancake on it's head gets more downloads every hour!"

And they would be right.

But think big picture - what if 50% of the downloaders actually open the zip file?

And what it 50% of them actually compile and run the product?

And what if 50% of them start looking into the source code and trying to write their own converter? 

And what if 50% of them actually succeed?

And what if just one of those people decides that their converter is worth sharing with the world?  How many people will that help?  Hunderds?  Thousands?  More?  Or maybe they decide to use the tool for consulting engagements - how many successful careers could be made using that tool?

In the migration space it does not require big numbers to make a big impact.

We've already impacted 100 people.  That is exciting to me.

How many more will download it tomorrow?  And the next day? 

Have you downloaded it yet?

Migration Toolkit Pre-Release published on CodePlex

Interested in writing a migration tool targeting (or synchronizing with) Team Foundation Server version control or workitem tracking?

Then check out the pre-release of the TFS Migration and Synchronization Toolkit on CodePlex!

http://www.codeplex.com/MigrationSyncToolkit

This drop is the complete source code for the migration toolkit and sample converters.  The VC sample is an application that will synchronize WSS document libraries with TFS version control.  Developers can check in doc changes with their code and managers can see the updated docs on the WSS server after the mirroring is done (and it works the other way too).

Oh - and this is still under active development (and has known issues) so if you have any feedback it's still early enough for it to be considered for inclusion in the final drop.

And to quote Matt ...

"Please note that this is a prerelease version of the Migration and Synchronization Toolkit. Several features are not complete and as testing has not completed, the code has the potential to contain bugs and overwrite data stored in TFS. Please be careful to use this only in a testing environment and not on live production data."

TFS Migration Toolkit Spec released for public review

As Matt noted on the migration blog a working copy of the toolkit specification has been released.  You can download it and leave feedback from this link:

http://msdn2.microsoft.com/en-us/vstudio/aa948851.aspx

 If you have feedback on migration scenarios for version control, work item tracking or other products (test, office, web, file systems, etc) please check it out and let us know.  We are still actively developing the toolkit and your feedback is very important to us!