Developer Support Team Foundation Server

Important events: Microsoft SQL Server 2005 SP2 end of life and Team Explorer 2008 Forward Compatibility Update (GDR) released

• Just wanted to remind everyone that on January 12, 2010, main stream support for SQL Server 2005 Service Pack 2 will end. In order to remain eligible for support, users of SQL 2005 will need to upgrade to SP3. Customers who do not upgrade will need to have custom support agreement to get continued support on SP2.  SQL Release Services has blogged about this here.

• The Team Explorer 2008 Forward Compatibility Update (GDR) is available now at this link. This is an update for the 2008 SP1 version of Visual Studio Team System Team Explorer and allows the 2008 SP1 version to work with the Team Foundation Server 2010. The update will allow teams to move forward and use the Team Foundation Server 2010 server even if part of the team continues to use the Team Explorer 2008 SP1 client.  For those of you on TE \ VS 2005, I am afraid you cannot use TFS 2010 yet, but there is a “Team Explorer 2005 Forward Compatibility Update (GDR)” expected around the RTM of TFS 2010 that will make the three compatible.  Further details on that are available here.

- Trev

“Exception message: Error in the application..” application event log error from TFS Service

Coop (Chris Cooper) bring us this little tidbit from a recent support case of his…

When looking at your application event logs, you may notice the following error from time to time:

Event Type:        Error
Event Source:    TFS Services
Event Category:                None
Event ID:              3071
Date:                     9/19/2009
Time:                     3:02:19 AM
User:                     N/A
Computer:          TFSSERVER
Description:
TF53010: The following error has occurred in a Team Foundation component or extension:
Date (UTC): 9/19/2009 3:00:19 AM
Machine: TFSSERVER
Application Domain: /LM/W3SVC/3/Root/Services-3-128977417369957288
Assembly: Microsoft.TeamFoundation.Server, Version=9.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a; v2.0.50727
Process Details:
  Process Name: w3wp
  Process Id: 2296
  Thread Id: 3796
  Account name: XYTECHLA\tfsservice

Detailed Message: TF200035: One or more errors occurred when Team Foundation Server attempted to synchronize with the following Active Directory identity: Team Foundation Valid Users. Number of errors that occurred: 1.
An error occurred when synchronizing the following Active Directory identity: Team Foundation Valid Users. Exception message: Error in the application..
For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.

When I saw it for the first time, I was perplexed. I only had a limited number of users and nobody had ever complained about access to the server. I had no trouble adding new users and I would only see this error once a day. So I began to dig a little deeper. Looking at the system event log, I saw the following error at the same time:

“A worker process with process id of XXXX serving application pool 'Microsoft Team Foundation Server Application Pool' has requested a recycle because the worker process reached its allowed processing time limit. “

So I looked at the application pool settings and noted that our application pool for Team Foundation Server was set to recycle every night at 3:00 am. The default setting is normally set to recycle every 1740 minutes (29 hours) but we had changed this reproducing another customers issue.

So here’s what’s going on. Synchronization with AD is performed by a background thread run by the TFS web application. We typically see this error due to scheduled restarts, because the restart occurs on the hour, and so does the sync. If you are setup this way, you will see it on the restart because just before the restart, identity sync was already in progress, then the process running identity sync is shut down. So this is basically a thread abort, that we report as “Error in the application”.

CSS TFS HOTFIX Report: September 2009

Between Sep 1, 2009 and Sep 30, 2009 we published the following HOTIFX articles\files and knowledge base articles:

Knowledge Base Articles

• 976184 - Empty groups still appear as available selections in work item fields in Microsoft Team Foundation Server

HOTFIX

• 975001 - TFS: Duplicate items created in Project Plans causing slow responses
• 974804 - Excel integration crashes when trying to 'get' new list of items

At time of writing these two HOTFIX articles have yet to be published and the fixes themselves are not yet available for download from Code.MSDN.com, though they should be soon. If you need them in the interim please open a support case with us. The case will be free of charge, we just need the case in order to track or work.

ACL and Group Membership change logging in TFS – what are your options?

As you may know, current versions of TFS do not offer any native method to log security changes (this story is not changing much in TFS 2010, either).  So what are your options for tracking TFS security changes now? Well, I’d like to present a few ideas based on some discussions I’ve seen internally this week…

• Manage your TFS security in Active Directory
  
  I have not researched it but I am fairly certain there have to be at least two AD monitoring tools out there that allow admins to track changes therein. We have long recommended that TFS group membership be managed through active directory groups, so once those groups are created and added to TFS, you maintain membership through Active Directory Users & Computer, or some other tool – not in TFS itself.
  
  
• Roll Your Own
  
  TFS is open and very extensible. It’s also basically a bunch of web services. Now that my be a huge oversimplification (as I’m sure anyone who’s ever supported it to any degree will attest), but the fact remains – the client interface to TFS and the intercommunication of TFS to itself is all handled via web services. As such, you are – for the most part, and given the proper rights – allowed to call these same web services for your own purposes. To that end may I present the IdentityChangedEvent, AclChangedEvent and DataChangedEvent TFS events. The first two were introduced in TFS 2005 and later deprecated by the third (DataChangedEvent) with the release of 2008. I won’t go into depth on their usage here, suffice it to say that these events should provide enough information to those wanting to monitor TFS ACL and group membership changes through their own home-grown apps. Brian Randell's MSDN magazine article on the TFS Eventing Service is a good reference to get you started here. Here is another by Mariano Szklanny. This one has some really cool code showing how to send emails based on certain WIT events, and a particularly useful tip on translating an AD user name into their email address!
  

Try as I might I was unable to find an example on the Interwebs of subscribing to/utilizing information from the DataChangedEvent. So, using the aforementioned samples in bullet two as a base, I created this simple one below. Basically it’s a web service which subscribes to DataChangedEvent on a TF server. It receives the XML from the event and if the  DataType == IDENTITY, passes the SeqId-1 to the GetChangedIdentities method of  GroupSecurityService2.asmx on the same server. It then writes what it gets back to a custom even log. In my testing it is triggered very well by adding a Windows user to a TFS group (for example). Once created and up on a web server - be it on the TFS AT itself or another IIS server - you will have to subscribe your web service to DataChangedEvent on your TF server. This is the command I used (one line), where…

• My CMD prompt was open to C:\Program Files\Microsoft Visual Studio 2008 Team Foundation Server\TF Setup on the TF server
• My TF server was named trevorh-wstfs08
• My custom listener web service was named ProcessDataChangedEvent.asmx, and was running on the TFS AT… just on another port (51235)

bissubscribe /eventType DataChangedEvent /address http://trevorh-wstfs08:51235/ProcessDataChangedEvent.asmx /deliveryType Soap /server https://TREVORH-WSTFS08:8081

This is not meant to be production ready by any means, of course. My intent here was to publish an example of how one might capture some of this identity change info from DataChangedEvent. Try it out and have a look in your Event Viewer on the machine hosting the web service. You may be surprised at the info you can get out of GetChangedIdentities  as you manipulate TFS permissions. The code is fairly well commented, but if you have any questions please post a comment of your own and I will try to help you out (no slagging my coding skills, please <g>).

Hope this helps!
- Trevor

Imports System.Web.Services

Imports System.Web.Services.Protocols

Imports System.ComponentModel

Imports System.IO

Imports System.Security.Cryptography.X509Certificates

Imports System.Net

Imports System.Net.Security

<System.Web.Services.WebService(Namespace:="TrevTools")> _

<System.Web.Services.WebServiceBinding(ConformsTo:=WsiProfiles.BasicProfile1_1)> _

<ToolboxItem(False)> _

Public Class ProcessDataChangedEvent

    Inherits System.Web.Services.WebService

    <SoapDocumentMethod( _

  "http://schemas.microsoft.com/TeamFoundation/2005" & _

  "/06/Services/Notification/03/Notify", _

  RequestNamespace:="http://schemas.microsoft.com" & _

  "/TeamFoundation/2005/06/Services/Notification/03")> _

  <WebMethod()> _

    Public Sub Notify(ByVal eventXml As String, _

                 ByVal tfsIdentityXml As String)

        ' Creates event log for this app. Only need to be run one per machine.

        ' SetupEventSource()

        ' Necessary in order to deal with self-served certificate on my TF server using HTTPS

        ' http://www.xtremevbtalk.com/showthread.php?p=1342414#post1342414

        ServicePointManager.ServerCertificateValidationCallback = _

            New RemoteCertificateValidationCallback(AddressOf CertificateValidationCallBack)

        Const logSource As String = "TFS_DataChangedEvent_Listener"

        Const logApp As String = "ProcessDataChangedEvent"

        Dim el As New EventLog(logApp)

        el.Source = logSource

        Try

            If eventXml Is Nothing OrElse eventXml = String.Empty Then Return

            Dim x As XElement = XElement.Load(New StringReader(eventXml))

            Dim DataType As String = x.<DataType>.Value

            If DataType Is Nothing OrElse DataType = String.Empty Then

                el.WriteEntry("DataType was empty")

                Return

            End If

            Dim SeqId As String = x.<SeqId>.Value

            If SeqId Is Nothing OrElse SeqId = String.Empty Then

                el.WriteEntry("SeqId was empty")

                Return

            End If

            el.WriteEntry("DataType " & DataType & ", SeqId " & _

                        SeqId & " received. Next event log contains data...")

            ' In my testing the SeqId returned is the next number that will be used (MaxSequence), 

            ' not the one that we're interested in, so subtract 1 from it before calling GetChangedIdentities()

            ' from /Services/v2.0/GroupSecurityService2.asmx

            SeqId = CType((CType(SeqId, Integer) - 1), String)

            If DataType = "IDENTITY" Then

                ' "MyTFS" is a Web Reference to https://trevorh-wstfs08:8081/Services/v2.0/GroupSecurityService2.asmx

                Dim GCI As New MyTFS.GroupSecurityService2

                '<http://geekswithblogs.net/ranganh/archive/2006/02/21/70212.aspx>

                GCI.PreAuthenticate = True

                GCI.Credentials = System.Net.CredentialCache.DefaultCredentials

                '</http://geekswithblogs.net/ranganh/archive/2006/02/21/70212.aspx>

                Dim ChangeInfo As String = GCI.GetChangedIdentities(SeqId)

                If ChangeInfo Is Nothing OrElse ChangeInfo = String.Empty Then

                    Return

                Else

                    el.WriteEntry(ChangeInfo)

                End If

            End If

        Catch ex As Exception

            el.WriteEntry(ex.Message, EventLogEntryType.Error)

        End Try

    End Sub

    ' Creates an event log. Make sure you have enough permission to do this.

    Private Sub SetupEventSource()

        Const logSource As String = "TFS_DataChangedEvent_Listener"

        Const logApp As String = "ProcessDataChangedEvent"

        If Not EventLog.SourceExists(logSource) Then

            EventLog.CreateEventSource(logSource, logApp)

        End If

    End Sub

    ' Necessary in order to deal with self-served certificate on my TF server using HTTPS

    ' http://www.xtremevbtalk.com/showthread.php?p=1342414#post1342414

    Function CertificateValidationCallBack( _

        ByVal sender As Object, _

        ByVal certificate As X509Certificate, _

        ByVal chain As X509Chain, _

        ByVal sslPolicyErrors As SslPolicyErrors _

            ) As Boolean

        Return True

    End Function

End Class

A Five Part Series: Troubleshooting TFS and SQL Reporting \ Analysis Services

Brett Keown brings us this series. We hope you find it helpful and contribute some suggestions for part 5.

-Trev
------------------------------------------------------

There are lots of moving parts to Team Foundation Server.  With depth knowledge necessary in Active Directory, SharePoint, Analysis Services, SQL Server, SQL Server Reporting Services, Visual Studio, Networking, IIS, etc… no one person can understand every aspect of the product or the implications of how making changes in one product will affect the other areas.  Supporting TFS is a team driven, multi-disciplined effort.  In support we find that it is much more efficient (and increases our customers satisfaction) if we scope issues to a root cause and then collaborate with different types of depth experts to resolve increasingly complex issues.  One product team we collaborate with frequently is SQL Server Reporting Services (SSRS).  Issues with SSRS and Analysis Services are common as these products will touch most of the technology areas Team Foundation Server is dependent upon.

With this in mind, I felt that it would be nice to provide our readers with some of the more common issues we see in Team Foundation Configuration Support.  The emphasis here is to address the more "common" issues we run into - things that typically can be resolved without finding a depth Subject Matter Expert (SME) to collaborate with.  To that end we begin today a series of five BLOG posts. The structure of these posts will begin with some fairly basic and straight forward break / fix type problems and move on to issues reported in our MSDN forums or previous cases.  In part five of this series I will take two topics that you our readers suggest and research them so that we may provide steps to guide you towards resolution.  Please comment any of the first four parts to have your ideas considered for part five.

Over the course of a month or so I will post regular updates to our blog which will cover the following topics:

Part 1:  An Introduction - Best Practices for rebuilding the TFSWarehouse

Rebuilding the TFSWarehouse is a task that every TFS Administrator will need to perform sooner or later.  What are the errors you may receive when going through this process?  What happens when you open Visual Studio in the morning and notice that the Reports node of Team Explorer has a "Red X" over its icon!?  Hopefully this post will provide you with the most common issues we run into with the TFS warehouse and the steps we take to resolve these issues. 

Part 2:  Incorrect BASE URL Setting in TFS for SSRS

An error we have seen often in support and on the MSDN forums references the Base URL to the reporting server for TFS.  What happens when this is wrong?  What errors do we receive which point to the problem?  Week two's post contains a very straight forward approach to resolving this type of issue.  Here is an error straight from the forums:

The path of the item "//Sites/MyTracker/Remaining Work" is not valid. The path must be less than 260 characters long and must start with slash. Other restrictions apply. (rsInvalidItemPath)

The second issue we will cover in part two deals with data flow and the steps you should take when you suspect that indeed, the data has stopped flowing from the Operational Store into the Data Warehouse.  There's a not so difficult process that can help you verify that you are running into this problem AND help you fix it.

Part 3:  Splitting SSRS and Analysis Services off the TFS App Tier to their own cluster

A configuration question we are beginning to see more and more is... "How do I split my Reporting and Analysis Services Servers off of the Application Tier and into their own clustered environment?"  While the issue itself is supported by our talented Installation and Setup team, I was fortunate enough to work with Arun Ramalingam recently who patiently and gracefully walked one of my customers through this process.  This post will document how they got the job done.

Part 4: Troubleshooting permission errors in SSRS

Ok, so your boss was working late gathering reports for that next business meeting with the company bigwigs... s/he sits down to begin work and receives an error,"The permissions granted to user "<username>" are insufficient for performing this operation." when attempting to access the reporting site.  The next call s/he makes is to you while you are at dinner with some friends.  How do you troubleshoot and resolve this issue as quickly as possible, look like the hero and get back in time for coffee & desert?  This post will contain the most common methods and resolutions for this error.

Part 5:  Audience participation; what would YOU like to see addressed here?

This one is up to you!  What two Reporting Services / Team Foundation Server issues would you like to see us address?  Please comment any of the posts in this five part series to have your ideas considered.  Reader support of your ideas will help us choose which two topics to cover.  Please keep in mind that we are focusing on "common" TFS / RS issues and not "one off" type incidents, though if you come up with one unique and interesting enough, we might just do it.

An Introduction - Best Practices for rebuilding the TFSWarehouse:

Ok, now that you we've outlined the scope of this series let’s get down business.  There are just a handful of issues we typically see with reports.  This post will cover the most frequent offenders, what the error messages actually mean and how we resolve the issue.  Most errors are self explanatory in nature (yes, believe it or not!) and most can be resolved fairly easily if you just know how.  For this post I will walk you through the process I would actually use with a customer who has called in with the problem statement, "I have a red x on reports in the Team Explorer."  Rarely do we see all of these issues in one call, but it has been known to happen.  So you have a Red X on reports, what now?  Here are the first things I will ask you when you call in:

What version of Team Foundation Server are you using including Service Packs?

What version of Team Explorer is being used on the client including hotfixes and power tools?

Which version of Visual Studio are you using... you guessed it, including Service Packs?

Which Operating System version are you using on each Server including Service Packs (are you beginning to see a theme here :))?

Which version of SQL Server are you using including Service Packs.  Is your installation of SQL Server in a clustered environment?

Where do your Analysis Services and Reporting Services instances reside?

Many TFS administrators will know this information right away.  The problem is that most of the time, it’s the clients that run into the issue.  If you do not know the answer to any of these questions, or if you are unsure... PLEASE let us know.  We can quickly assist you in answering each of these questions.  If you give us the wrong answer we can spend many hours (and many dollars) troubleshooting an issue that we may have been able to resolve in five minutes.  You will see this below.  You can get this information quickly as well.  Simply download the Microsoft Visual Studio Team Foundation Server Power Tools.  In the Power Tools package you will have an opportunity to install the Best Practices Analyzer which will poll your installation for all of this data and put it into a handy tree structure which can be reviewed by you or uploaded to support.  I don't want to ruin the surprise, so I won't get into EVERYTHING that the BPA scan will tell you, but it is the single handiest tool we have for troubleshooting configuration issues.  Download your version here:

For TFS 2005

http://www.microsoft.com/downloads/details.aspx?familyid=7324c3db-658d-441b-8522-689c557d0a79&displaylang=en

For TFS 2008

http://www.microsoft.com/downloads/details.aspx?familyid=FBD14EEA-781F-45A1-8C46-9F6BA2F68BF0&displaylang=en

Figure 1: A broken TFS Installation I use for testing

This tree view contains data on your installation, versions, server locations, permissions, product conflicts, event logs, application pools, etc.

We ask these questions for a few reasons.  We'd like to understand your infrastructure before we begin working and there is a common call we get for this "Red X" on reports issue.  Essentially this error message means that Team Foundation has requested a list of children from Reporting Services.  For whatever reason, we can not retrieve this list of children.  If you have a mixed environment of 2005 and 2008 clients, things will be humming along just fine, and then one day you come into the office to find that all of your TFS 2005 clients have this issue.  What gives?  Chances are you're IT group updated to SQL 2008 last night (or more likely, over the weekend).  Well, what do you do now?  TFS 2005 clients are not supported with SQL Server 2008.  This will not work and no amount of mucking around with it will get it to work.  You have two options really.  1.  You can update all your clients to the 2008 version of TFS, or; 2.  You can give your clients a method for obtaining the data they need outside of TFS.  The best way to do this is to just browse to the reporting site http://<servername>:80/reports.  Once there, choose the report they would like to run.

When we are sure you have the correct product versions, the very next question will be "Has it ever worked?".  The Best Practices Analyzer (BPA) will catch most issues with reports that have never worked.  So let’s say you browse to the reports site but are having some very bad luck.  The first issue you encounter is this error: 

An error has occurred during report processing. (rsProcessingAborted) Cannot impersonate user for data source 'TfsOlapReportDS'. (rsErrorImpersonatingUser).  Log on failed. (rsLogonFailed)

What now?  Well most likely you are having a permissions issue (rsLogonFailed).  You can verify this by checking the Security Event logs for issues of type "Audit Failure".  Well how do you know what permission failed and where it failed from?  Go back to the error message.  It states that it ran into a problem while attempting to impersonate a user when connecting to the data source 'TfsOlapReportDS'.  Let’s take a look at that data source and verify the credentials. 

Figure 2: TFSOlapReportDS Properties Window

I am sure I have the right password in here (as we all are <g>) but I'll type in my existing credentials anyhow, then hit "apply".  No need to reset IIS here, go ahead and try to run the report again.  It seems that we've gotten through this error, because now... we have a new one.  Ok, onto the next issue.

An error occurred during client rendering.  An error has occurred during report processing. Cannot impersonate user for data source 'TfsReportDS'.

Log on failed.

Well, it makes sense that if changing the password for TFSOlapReportsDS fixed our previous problem, than we have the same issue occurring with TFSReportDS displayed with a slightly different message.  This error references the same "Log on failed" message and also states "Cannot impersonate user for data source 'TFSReportDS'.  I go through the same process of typing in my username and password on the data source properties page in http://<servername>/reports for TFSReportDS and voila, it works!  Wonderful, the report frame has come up, but wait... There is no data.  I look at the timestamp at the bottom of the report and see that the TFSWarehouse hasn't been processed in over a week! That makes sense as the account credentials were invalid.  Well, we know it’s not a data source access issue now.  Maybe we were having some more widespread errors that were masked by the first few problems.

When I browse to my Report Manager for TFS, I can see that the report was generated correctly.  The timestamp for "Report Generated" reflects the time / date the report was run.  But there is no data below.  When I look just to the right of the first timestamp, I see that the "Last Warehouse Update" was over a week ago.  At a minimum, we update the warehouse hourly using the default settings.  Yours may be much different depending on how customized your implementation of TFS and report processing are.  So if we aren't updating the reports, we could try to go into SQL Server Management Studio, connect to Analysis Services and process the warehouse, but if we have a problem in there already, most likely processing the warehouse isn't going to do much to help with our current situation.  Instead, let’s rebuild the TFSWarehouse. Since its data is taken from other databases it can be rebuilt even if you don't have a backup; that is, assuming your other databases have good data.  Take a backup anyhow to be safe, and then run "The Big One".  If you have read our blog for any amount of time you will most likely be familiar with the term "The Big One".  It’s our nickname for a rebuild of the TFSWarehouse.  We call it The Big One due to the length of the command entered from the command prompt. 

IMPORTANT NOTE: Before continuing with the rebuild of the warehouse, PLEASE read this post (it’s not as serious as it sounds, but you should be aware):

Use of “SetupWarehouse.exe –rebuild…” can cause data loss in Team Foundation Server 2008

“During warehouse sync, the Build adapter uploads the build information to the TfsWarehouse relational database. When the upload of data is completed the tbl_ServerStatus table is updated with a watermark of the last uploaded build. Each night there is a SQL job (TfsBuild Team Build Administration Job) which runs and based on the information from the tbl_ServerStatus table, removes all unnecessary nodes from builds that have already been uploaded to the TfsWarehouse relational database.

That means that if  you rebuild the warehouse, this information will be missing from the build op store hence it will not be uploaded by the adapter in the TfsWarehouse relational database.

The obvious question is:  Will this rebuilding the warehouse affect the data from the build reports?

The short answer is no. The build related reports that come with Team Foundation Server display compilation summary information at the root node level. As a result, even though rebuilding the warehouse removes compilation summaries for child nodes that have 0 errors/warnings and static analysis errors/warnings, it will not affect the data displayed in the reports.”

Once your backups are complete, go ahead and open a Visual Studio command prompt.  Next, type in the following command replacing <bracketed> values with your own accounts and server names:

“C:\program files\microsoft visual studio 2008 team foundation server\tools\setupwarehouse.exe” /setup /install –n –s “<DATA TIER SERVER>” –d “TFSWarehouse” –c “C:\program files\Microsoft visual studio 2008 team foundation server\tools\warehouseschema.xml” –ra “<DOMAIN\REPORTING SERVICES ACCOUNT>” –a “<DOMAIN\TFS SERVICE ACCOUNT>” –v –l

Here's an example with real values:

“C:\program files\microsoft visual studio 2008 team foundation server\tools\setupwarehouse.exe” /setup /install –n –s “BRETTKETFSDT” –d “TFSWarehouse” –c “C:\program files\Microsoft visual studio 2008 team foundation server\tools\warehouseschema.xml” –ra “Redmond\bktrsreports” –a “Redmond\bktfsservice” –v –l

If you're lucky or just pay very close attention to the details, this will work great and you will receive a message indicating that the update has completed.  If not (and most people fall into this category) you will receive one of a few errors.  What errors can be returned when running this command?  What do they mean?  For this blog post I intentionally made some mistakes so that we can review some of the types of errors we may run into.  Here's the first error returned:

An unexpected error occurred: Error encountered when creating connection to Analysis Services. Contact your Team Foundation Server administrator.

This could be a few things, but if you are looking for common issues, most likely you have a typo in your Data Tier server name. You can find the correct name by opening SQL Server Management Studio and connecting to Analysis Services.  Once there, right click on the TFSWarehouse and view the connection properties.  In the product properties window you will find the server name listed. So I correct the error (SHIFT + TAB in the command window will bring up your command again, then use the arrow keys to go back and correct the problem) and run the command again:

NOTE: A great way to avoid typo’s is to use a .CMD file which you can create in notepad. It’s much easier to edit a .cmd file in notepad, then to edit from the command prompt directly. Bing it if you would like to learn how; there are lots of examples available on the internet.

C:\Users\brettke>"c:\Program Files\Microsoft Visual Studio 2008 Team Foundation Server\tools\setupwarehouse.exe" /setup /install -n -s "BRETTKE082008" -d "TFSWarehouse" -c "c:\program files\Microsoft Visual Studio 2008 Team Foundation Server\tools\warehouseschema.xml" -ra "redmond\bktfsrept" -a "redmond\bktfserv" -v -l

Dangit, we have another error:

An unexpected error occurred when connecting to the SQL Server. Check that your SQL Server is up and you have sufficient privileges on it: Windows NT user or group 'redmond\bktfsrept' not found. Check the name again.

To test that the account is good, I go over to another machine and try to log in with the credentials I provided to TFS.  This is the single most efficient way to test an account name, its password, domain authentication for a single account, etc.  When attempting to log in I am denied access to the machine.  Well, it must be a problem with the account.  So I look at the account name again and realized that I had a typo there as well.  If I am certain the account name is valid, I would begin checking for a valid password.  Once I fix this typo I am able to successfully rebuild the warehouse and receive the message:

Setup warehouse Completed Successfully.

Great!  Now I go back into the Report Manager and run my report.  As my daughter would say (Check the Dora reference) ... "Awww Mannnnn! Another error!” This time right in the report window we see:

An error has occurred during report processing. (rsProcessingAborted)

Query execution failed for dataset 'IterationParam'. (rsErrorExecutingCommand)

For more information about this error navigate to the report server on the local server machine, or enable remote errors

This error is telling us that we have rebuilt the warehouse, but we have not repopulated it with fresh data! Here are the steps to refresh the data in TFSWarehouse.

1.  Browse to http://<servername>:8080/Warehouse/v1.0/warehousecontroller.asmx

2.  Click on "GetWarehouseStatus", and then click "Invoke".  It should return "Idle".

3.  Now, let’s test it to make sure it’s really running.  Go back to the ControllerService page and choose Run". 

4.  On the next window, click "invoke".  This should return a value of "true".

5.  This next step can take awhile depending on how much data you're processing.  Now that the adaptor is running, click on "GetWarehouseStatus" every 10 minutes or so to check where it is.  There are a few different states that can be returned:

• Running Adaptors means we are getting started on processing.
• Processing OLAP is just that.  TFSWarehouse is now processing the OLAP database and repopulating the cube
• Idle lets you know we are done.    

NOTE:  if changes are made to the schemas during the "processing olap" phase, the status will return to running adaptors.  Its ok, this is as it should be.  Just keep letting it run until it returns idle.  Now go into the Report Manager... and finally... it’s done.

 Report Generated: 9/16/2009 4:04:05 PM by REDMOND\brettke; Last Warehouse Update: 9/16/2009 4:02:30 PM

In part 2 we will be discussing data flow problems and the Base URL!

Brett

Getting TFS support faster from Microsoft: CSS structure revealed

Wendell brings us a post today about the way we are arranged in Microsoft TFS support in the hopes that it will help you route to the proper team and get answers faster. Enjoy!

-Trev

++++++++++++++++++++++++

I was looking through some of our support cases a couple of days ago.  It struck me that sometimes a solution can be delayed because the description of the issue was inaccurate or incomplete and the resulting support case was routed to the wrong team.  I hope the following information about our internal organization will help you, our customers, understand why an accurate description will speed up our response.

The TFS Support Organization is separated into four support areas handled by three different teams of engineers.  We refer to these teams as “Setup”, “Config”, and “Test” and “Build”.  The Test and Build support areas are handled by the same team of engineers, but are separately identified for internal purposes. 

When the description of the issue is accurately captured the case can be immediately routed to the correct assistance.  When the case description is incomplete or inaccurate, the case may be sent to the incorrect team causing unnecessary delay.

Here are the descriptions the teams provide to assist our internal call routing specialists:

Setup - “DSC Install and Setup” - Our team handles the installation and setup of Team Foundation Server (TFS).  The setup team handles moves, upgrades, restores and disaster recovery scenarios as well.

Config - “DSC TFS” - Our team handles the administration and management of Team Foundation Server (TFS) versions 2005 and 2008.  This includes performance issues, connectivity, event log errors, security, configuration, failed project creation (unless TFS has just been installed\upgraded and project creation has never worked – that goes to ‘setup’).

Build - “DSC Team System” -  The Visual Studio Team System team handles support for the Team Build portions of TFS.

Test - “DSC Team System” - -  The Visual Studio Team System team handles support for Visual Studio Team System for Software Developers and Software Testers, which includes support for Unit Testing, Web Testing, Load Testing, Generic Testing, Manual Testing, Code Analysis for C/C++, Code Analysis for Managed Code, and for Profiling Application. This team also support the Application Center Test technology (ACT) and programming against the TFS object model.

So please help us help you and add at least a few key words to help us get you the right support the first try.

CSS TFS HOTFIX Report: Jan – Aug 2009

Between Jan 1, 2009 and Aug 31, 2009 we published the following HOTIFX articles\files. I am basing this list on the criteria that the fix was uploaded to our HOTFIX server between those dates:

• 959168 - FIX: The destination file may change if you use the "tf merge" command together with the /baseless and /discard options on a computer that has Visual Studio 2008 Team Foundation Server SP1 installed
  
• 960965 - FIX: Error message when you run VSSConverter.exe to migrate a Visual SourceSafe database to Visual Studio 2008 Team Foundation Server: "BaseExceptionMessage: An item with the same key has already been added"
  
• 961267 - FIX: The application crashes when more than approximately 50 users connect to Visual Studio Team System Web Access 2008 SP1
  
• 967583 - FIX: Error message when you move a folder and then check in the changes in Source Control Explorer in Microsoft Visual Studio 2008 Team Foundation Server SP1: "TF30073: Please contact your administrator"
  
• 962968 - FIX: You receive one or more error messages when you use a local account to install Microsoft Visual Studio 2008 TFS or when you try to add a local account to Visual Studio 2008 TFS
  
• 967483 - A hotfix that enables Team Foundation Server 2008 to work with Expression Studio 3
  
• 967690 - FIX: Performance of testing decreases after several hours when you run load tests by using team system load testing tools in Microsoft Visual Studio 2008 TFS
  
• 969210 - FIX: Error messages in various reports in Visual Studio 2008 Team Foundation Server Service Pack 1 with SQL Server 2008 Reporting Services
  
• 969868 - FIX: When you run a build of a project that is created in Microsoft Visual Studio 2008 Team Foundation Server SP1, some changesets are not detected as expected in the GenCheckinNotesUpdateWorkItems task
  
• 940295 - FIX: The check-in process may be very slow when you use the Source Control feature in Visual Studio 2005 Team System
  NOTE: This was just localization of the same fix – originally published in September of 2007 – into Japanese. 940295 is now available in ENU and JPN.

Interesting support case – Visual Studio solution prompts for entire SCC check-out on open

Today we hear from Brett Keown about an interesting support case he had recently, with some great details about CSPROJ file structures, etc. Enjoy!

Trevor

+++++++++++++++++++++++++++++

I had some questions yesterday from a customer about an issue in which every time they opened a solution file locally or from their TFS Server, they were prompted to check-out the entire code tree.  It turns out that the resolution is fairly straight forward though finding it was a bit tough (thank you JH!  You know who you are :-) )!  It turns out that my customer’s csproj files had been modified either manually or by a tool to add values that should not have been there.

In project files there are four values that we use to control this behavior.  They are SccProjectName, SccLocalPath, SccAuxPath and SccProvider.  The value for each of these "should" be "SAK" (with the quotation marks). If you take a look in a project file (in my case, csproj) you will see an SCC section that looks like this:

        SccProjectName = "SAK"
        SccLocalPath = "SAK"
        SccAuxPath = "SAK"
        SccProvider = "SAK"

The settings in the project files override the settings in MSSCCPRJ.SCC which is created by MSSCCI compliant source code control providers.  The SAK essentially says… “Ignore me and go look at the MSSCCPRJ.SCC file for this information”.  Here is what my customer had.  Ouch!

<SccProjectName>%24/BLAH111/GreatProject/Dev/Feature1/Desktop</SccProjectName>
<SccLocalPath>..\..</SccLocalPath>
<SccAuxPath>http://CoolTFSServer:8080</SccAuxPath>
<SccProvider>{4DB58AB2-18RA-4F8K-95U4-32PDF27L184R}</SccProvider>

The MSSCCI portion of the Visual Studio Development Environment will read these settings and never get the info contained in MSSCCPRJ.SCC.   Because we can't find the information that TFS (or VSS) is expecting to see, MSSCCI believes that it has to rebind the solution and its dependent files back to the correct location so we are prompted to checkout / checkin on every edit, get, or history of the files.

The fix for this issue was to use REGEX (regular expression editor) to walk all the csproj files and replace the incorrect values with “SAK” for each line.  MSSCCI will then ignore the project file settings and go look for the information in the MSSCCPRJ.SCC file.

Those of you that have used Visual SourceSafe for a while most likely know the name "Alin Constantin".  He's been working with this community in a very transparent manner for years leveraging his experience as one of the product’s (Visual SourceSafe) developers (he's a heck of a nice guy to boot!).  Alin has written a great document which discusses MSSCCPRJ.SCC and all the other version control files here:

http://alinconstantin.members.winisp.net/webdocs/scc/SccFiles.htm

I'd like to restate what Alin says very clearly... this is important now...

DO NOT DELETE THESE FILES!

Here' is Alin's description of the MSSCCPRJ.SCC file:

MSSCCPRJ.SCC - This file is created by MSSCCI-compliant source control providers

This file is created by source control providers like Visual SourceSafe if they can track the location of certain files in the database. A provider creates this file if it can store the AuxPath and ProjectName bindings for a file.

To help the source control integration in Visual Studio, a source control provider needs to track the location of solution and project files. When the provider is able to track the source control location of a project file, the source control bindings of that project will not be written in the project and solution file (dummy 'SAK' strings will be used instead as a signature in this case). This helps scenarios like Branching, when the project will automatically be bound to the branched location without the user having to use the File/SourceControl/ChangeSourceControl dialog to retarget the new branch. Also, creating mssccprj.scc may help keeping the source control integration working even when the users don't use source control integration properly (when they Get projects using VSS standalone instead of using the integration commands OpenFromSourceControl)

In SourceSafe the list of files that will be tracked can be configured using SourceSafe Explorer in Tools/Options/FileTypes/CreateSCCFile.

Here is an example of a mssccprj.scc file content:

      SCC = This is a Source Code Control file

      [ClassLibrary1.vbproj]

      SCC_Aux_Path = "D:\Program Files\Microsoft Visual SourceSafe 6.0"

      SCC_Project_Name = "$/Solution1/SubFolder/Solution1/ClassLibrary1", PBAAAAAA

For a description of mssccPrj.scc file format you can consult the most recent MSSCCI specification found here:

http://alinconstantin.members.winisp.net/webdocs/scc/MSSCCI.htm

TfsDeleteProject.exe thwarted by Windows SharePoint Services permissions

The other day I was cleaning up one of my Hyper-V machines. I had let Jim use it for a project creation test and since he was done, I wanted to whack that TF project to regain some space. So, I fired off this command, like you would…

TfsDeleteProject /server:https://TREVORH-WSTFS08:8081 "JimSau Project"

Failure…

TF30063: You are not authorized to access trevorh-wstfs08.8081

Interesting, considering I am a machine admin, TFS admin, and more on the machine. My assumption was I had deity-like powers on the box and this was just the result of a typo. A quick check later though and I found that it was not a typo, but a valid error.

I won’t go into the gory details on this, since there are quite a few forum and BLOG posts about it already. I will say that in my case the problem was lack of rights in WSS to the project portal I was trying to delete. I will also say that I found the feedback from TfsDeleteProject to be insufficient, and have entered a bug against it for Dev 10 – asking that it provide more feedback on the exact permissions failure. So a word to the wise on this: if you encounter this error, or one like it, when trying to use TfsDeleteProject to delete a Team Project, make sure you have sufficient privileges in all the areas which comprise it: SQL RS, WSS, and TFS.

-Trev

Hyper-V Backup / Restore when upgrading to Windows Server 2008 R2

I use Hyper-V and Windows Server 2008 R2 RC right now to host all my TFS reproduction machines, on which I work customer issues. It’s time to move that Hyper-V box to RTM, since it is available to us internally. My Hyper-V server is set up thusly (three physical HDDs):

C Drive: OS
D Drive: Page file
E Drive: Hyper-V files (VHDs, ISOs, snap shots, etc.)

Since I store nothing on C, this makes for a simple FFR (FDisk, Format, Reinstall). I don’t need to remember any settings or copy off any files or what have you (I do this with all my PCs so I can whack them fast if need be. The new “libraries” feature in the Windows 7 code base is making this even easier now). The VPCs for Hyper-V would be on E in both the old and new configuration, so I figured it would be a simple matter of just leaving all the files be and just add them back to the new Hyper-V instance after the new OS was installed. Not so fast! Unlike Virtual Server 2005, it’s not a simple matter to just nuke the Hyper-V install (via the OS reinstall) and then re-add VPCs into the new Hyper-V instance. You need to export\import in order to retain settings and snapshots. This kind of threw me for a loop because it implies that I need a 4th drive – one to export all my Hyper-V files off to - since I don’t have enough space on all my drives to duplicate my current VPCs, what with all the snapshots and drives in use.

So, being who I am I complained about this on an internal discussion list for Hyper-V. My squeaky wheel was given this grease:

http://blogs.technet.com/virtualization/archive/2009/05/29/hyper-v-r2-import-export-part-6-so-what-happened-to-configuration-only-export.aspx

The recommendation being that I could do a config-only export of the Hyper-V files. Unfortunately, it didn’t work. The export worked but on import I was given an error from the script:

Import failed. Number of parameters supplied for resource paths are not consistent. You should have a SourceResourcePath entry for every CurrentResourcePath entry in import structure.

I also tried using the Hyper-V Manager to import the files I exported using the script. Same error. Sooooo, I just gave up and brought in one of my 1TB drives to export to. All is well now. I am not sure what went wrong with the script(s), and I am by no means disparaging the authors, but I just didn’t have time to troubleshoot it; I needed to move on.

Have a similar experience? Have a recommendation for doing this better? Comment below - I’d love to learn. :-)

-Trev

PS: If you have trouble grabbing nicely formatted copies of those scripts from the Virtualization blog like I did (all my copy\paste attempts from that web page resulted in loss of carriage return\line feeds - I got a jumbled mess <g>), let me know via the “Email” page and I’ll send you “good” copies of them.

Hyper-V Remote Administration from Windows 7?

As you may know from this earlier post, I admin my multiple TFS Virtual PCs (which run on Hyper-V) from a Vista PC. Well last week I replaced that Vista PC with a Windows 7 Enterprise one. I tried to install the Hyper-V Remote Management Update for Windows Vista on my new Windows 7 PC and was denied (“this update does not apply to your system”, or the like). I thought I was done, but turns out there is an option for this in Windows 7 as well. This may be old news to you, but if not you can admin your Hyper-V PC remotely from Windows 7 by doing this:

1. Download the Windows 7 RSAT from here and install it (link updated Aug 12, 2009 to point to RTM version).

2. Open Programs and Features, select “Turn Windows Features on or off” and then drill down to…

   Remote Server Administration Tools
    \- Role Administration Tools
       \- Hyper-V Tools

2. Check off the Hyper-V Tools, let it do it’s thing. When complete you’ll have the Hyper-V Manager available in your Administrator Tools.

BTW, RSAT for Windows 7 doesn’t just give you remote Hyper-V management, but a whole host of other options as well.

HTH,
- Trev

Troubleshooting Team Foundation Server connection problems with Fiddler

Chris Cooper comes to us today with the story of a TF server, a client, and the Fiddler who saved the day….

--Trev

-----------------------------------------------------------

The other day I had a customer that had the dreaded Red-X on his documents node for one of his Team Projects in Team Explorer. The rest of the Team Projects did not have the Red-X and were fully accessible through Team Explorer. So the first thing I checked were the SharePoint permissions granted to the users for that particular Team Project (TE Documents node = SharePoint). That all checked out. I then had him browse the site in Internet Explorer – worked just fine too.

So the next place I went were the IIS logs. Once again, everything appeared normal – I could see the handshake and the 200 response. Well I guessed then it was time to turn on some logging to try to figure out just what the heck was going on.

The first thing I thought of was a system.net trace. You can enable this in the Devenv.exe.config file (usually in C:\Program Files\Microsoft Visual Studio 9.0\Common7\IDE) by adding the following to the file (make sure you close Visual Studio first):

<system.diagnostics>
       <trace autoflush="true" />
           <sources>
               <source name="System.Net" maxdatasize="1024">
                   <listeners>
                       <add name="MyTraceFile"/>
                   </listeners>
               </source>
             <source name="System.Net.Sockets" maxdatasize="1024">
                   <listeners>
                       <add name="MyTraceFile"/>
                   </listeners>
               </source>  
          </sources>

            <sharedListeners>
               <add
                 name="MyTraceFile"
                 type="System.Diagnostics.TextWriterTraceListener"
                 initializeData="c:\System.Net.trace.log"
               />
           </sharedListeners>
           <switches>
               <add name="System.Net" value="Verbose" />
             <add name="System.Net.Sockets" value="Verbose" />
           </switches>
   </system.diagnostics>

We opened Visual Studio and Team Explorer, which tried to connect to and refresh the Documents node, and then I viewed the trace results. What was interesting was that I could see the successful call to list libraries and list items but nothing was returned from the server.

Strange.

So on to Fiddler. You can get fiddler from the following site:

http://www.fiddler2.com/fiddler2/

This finally did it for me – this is what I saw in the logs:

<List DocTemplateURI=”/sites/My Project/test1/template.doc” DefaultViewURI=””

One of the document libraries (“test1”) in the problem project had a missing “DefaultViewUrl” setting.  TFS parses for all the DefaultViewUrl’s in all the document libraries in order to successfully connect to the Documents  in the project so one missing setting can cause this issue.  The simplest way to resolve this issue is to delete the “test” document library using SharePoint Site Settings for the Project site.  This is only a viable solution if the documents that are in the “test” library are not needed.  If they do need to be saved they can be copied out then added to a new document library created after the “test” library is deleted.

I had one other experience recently using Fiddler that I’d also like to tell you about. We had another customer, similar issue, could browse the portal but had a Red X on the documents node. I went through the same steps as before, but still did not have an answer.

So back to Fiddler. Looking at the log I saw that there were about 75 document libraries on this site.  The Documents node in Team Explorer 2005/2008 only supports 64 libraries, then it fails.  What happens is that 2005/2008 clients try to populate the contents of all the document libraries on initial connection to the project.  The client issues 75 requests to the server, one for each document library, each on a separate threadpool thread, then it attempts to wait for them all to complete.  One problem with this is that you can only wait on 64 handles at a time, so it fails.

The only workaround is to decrease the total number of document libraries on the site, or wait for the 2010 TFS client.

So the morale of the Red-X on  Documents node in Team Explorer story… this can fail if:

1. The user doesn’t have at least read access to all of the document libraries.

2. There are more than 64 document libraries (TFS 2005\08).

3. One or more of the document libraries has an incorrect or missing ‘DefaultViewUrl’, which can result in the TE not being able to retrieve the path name for a document library. 

4. Any of the subsequent call to get the contents of a document library fails.

5. Sun spots. ;-) [ed]

Determining the number of Team System Web Access Users

We have documented some scalability limitations for Team System Web Access in a white paper called Team System Web Access 2008 Scalability Limits. The white paper contains some configuration recommendations and limitations for TSWA.

One of the recommendations is to limit the number of concurrent users to a number below 100. Unfortunately, TFS and TSWA do not provide an easy way to measure TSWA user load.

With this in mind, I have created a report that will provide this information.

This report retrieves data from the tbl_Command table in the TfsActivityLogging database, in the same way as the reports in the Grant Holliday's TFS Performance Report Pack. The TSWA report requires the TfsActivityReportDS data source as listed in Grant's instructions. The best approach is to install Grant's reports, and upload the attached TSWA report to the same folder in Reporting Services. Be sure and read my blog post here for some additional instructions, and for SQL Server 2005 Reporting Services-compatible reports if you are using SSRS 2005.

The TSWA Usage report takes four parameters. These are Start Date, End Date, TSWA Instances, and IP Addresses. Because you may have multiple TSWA instances, the report allows you to filter by the instance name (as reported in the UserAgent column of tbl_command) and IP address (the IP of the computer where the TSWA instance is running). If you have multiple TSWA instances of the same version running on the same computer, we currently don't have a way to differentiate between these instances. The TSWA Instances and IP Addresses parameters are multi-select, with all values selected by default.

There are 3 tables displayed in the report. These are Unique Logins By Instance / IP, Total Logins By Instance IP, and User Logins By Instance / IP.  The first is the number of unique logins from each TSWA instance. Here multiple logins by the same user are counted once. This is the one you should use to measure number of users. The second report shows the total number of logins from each TSWA instance (i.e. duplicates are counted), and the third the number of logins broken out by individual users.

Enjoy! 

-Jim

SSL on TFS 2008 + Windows Server 2008 + IIS 7

MSDN has long had a document on Setting up Team Foundation Server to Require HTTPS and Secure Sockets Layer (SSL). It is geared specifically to Windows Server 2003 and IIS 6 though, as the OS and IIS platforms. Ruidong Li (a senior support engineer on our TFS config support team) went through that document recently and updated it with Windows Server 2008 / IIS 7 in mind. It is attached here for your reference. Please let us know if you find it useful or have any comments\corrections.

Windows 7: Be first. Save half.

You know I love me some Windows 7, and if you’re like me, you’re going to love this too…

Pre-order Windows 7 today at Microsoft Store. You'll get it for at least half off and be one of the first to get it. Hurry, quantities are limited.*

It's pretty simple

Windows 7 is coming on October 22, 2009. Here's an easy way to get it fast and save a bundle: Pre-order a Windows 7 Home Premium Upgrade for $49 or a Windows 7 Professional Upgrade for $99 at Microsoft Store.  That's about half off the estimated retail prices.

Want more info?  Go to the Windows team blog and find out all the details. 

*The offer begins on June 26, 2009 and will continue while supplies last, or until July 11, 2009, whichever comes first.

Click here for the deal (link live ~ @ 9pm PT June 25th, 2009)