Team Foundation: Alerts missing subject line?

Team Foundation creates alerts that notify users and/or web applications of certain events that have occurred. Some of these alerts are used internally by Team Foundation (e.g., notifying another TF component of an important change to group membership data). Most are used by end users to receive notification of work item changes, checkins, and build activitites.

Pete Sheill and others have discussed alerts in the past. Recently, we became aware of a problem that manifests itself through the Team Foundation eventing mechanism which includes alerts.

When the alert subject line is blank, this often means that one of the assemblies used to process alerts prior to sending isn't working properly. In some cases, these assemblies fail to load and an event log message will be written.

There are 3 assemblies that process alerts prior to sending to the subscribers

• Microsoft.TeamFoundation.Build.Server.EventFilter.dll
• Microsoft.TeamFoundation.VersionControl.Server.EventFilter.dll
• Microsoft.TeamFoundation.VersionControl.WorkItemTracking.Server.EventFilter.dll

These assemblies are loaded by the Services web application when it starts. This web application matches events with subscription data created from Visual Studio or using the command line tool BisSubscribe

We've found that these assemblies do not load when the configuration settings for that web service have been changed using the ASP.NET tab of the web site properties. You can reach this by navigating to the Team Foundation web site from inetmgr, selecting Properties and then choosing the ASP.NET tab followed by selecting Edit Configuration.

This causes an attribute to be added to the config element of the corresponding web.config file. E.g., if you alter the Services configuration, installPath\Web Services\Services\Web.config is altered.

Before making the configuration change, the web.config files look like

<configuration>

<!-- Team Foundation/Web application values -->

</configuration>

Following a configuration change using  Edit Properties:

<configuration xmlns="http://schemas.microsoft.com/.NetConfiguration/v2.0">

<!-- settings -->

</configuration>

To restore behavior, use an editor or Notepad to remove the namespace attribute. Note that you do not need to run iisreset following this change.

The underlying problem is a known issue and is being addressed.

Error Reporting -- Does anybody review those Watson reports?

I'm part of the group that reviews Watson reports for Team Foundation. On a weekly basis, we meet and review all Watson reports received since the last review period. Each report is first checked to see if it is for a known problem. Many reports are for issues that we have either discovered from internal users or from a previous Watson report. If the issue has not been seen before, then we will gather information and assign a bug to the responsible developers. We also track Watson bugs and make sure that the bug is treated appropriately by the developers.

Watson crashes represent a concrete occurrence of an issue -- sometimes, we find that the issue in a Watson report is not something that we can act on (e.g., a time-out condition) but we can handle better by informing the user or using an alternate algorithm.

We treat problems reported through Watson seriously and I want to assure you that Team Foundation Watson reports that you permit to be sent to Microsoft will be treated appropriately and timely.

Let me know if you have any concerns about sending Watson reports for Team Foundation.

(Updated) Sample: Version Control RSS Feed

I have updated the sample RSS feed; if you're using Beta 3, you should use the updated feed posted below. The sample was first posted here.

The sample below allows you to subscribe to a feed which generates information for a specific user. Append ?user=domain\alias to the subscription path.

<%@ Page Language="c#" %>
<%@ OutputCache Duration="20" Location="Server" VaryByParam="state" VaryByCustom="minorversion" VaryByHeader="Accept-Language"%>
<%@ Assembly Name="Microsoft.TeamFoundation.Client, Version=8.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a" %>
<%@ Assembly Name="Microsoft.TeamFoundation.VersionControl.Client, Version=8.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a" %>
<%@ Assembly Name="Microsoft.TeamFoundation.VersionControl.Common, Version=8.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a" %>
<%@ Assembly Name="Microsoft.TeamFoundation.VersionControl.Common.Integration, Version=8.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a" %>

<%
// "Copyright © Microsoft Corporation.  All rights reserved.  These Samples are based in part on the Extensible Markup Language (XML) 1.0 (Third Edition) specification Copyright © 2004 W3C® (MIT, ERCIM, Keio. All rights reserved. http://www.w3.org/consortium/legal/2002/copyright-documents-20021231."
%>

<%
//This posting is provided "AS IS" with no warranties of any kind and confers no rights.  Use of samples included in this posting is subject to the terms specified at http://www.microsoft.com/info/cpyright.htm.
%>

<%@ Import Namespace="System" %>
<%@ Import Namespace="System.Collections" %>
<%@ Import Namespace="System.IO" %>
<%@ Import Namespace="System.Web" %>
<%@ Import Namespace="System.Xml" %>
<%@ Import Namespace="Microsoft.TeamFoundation.Client" %>
<%@ Import Namespace="Microsoft.TeamFoundation.VersionControl.Client"%>
<%@ Import Namespace="Microsoft.TeamFoundation.VersionControl.Common" %>

<%
// Generate an RSS feed for Team Foundation Source Code Control checkins
//
// Note: Only one request per "pester interval" from each system will be honored. See
//       pesterInterval below.
//
// This feed returns information about the most recent N checkins. See maxCheckinCount
// below.
//
// Invoking this page without any parameters returns information about all Team
// Foundation checkins up to the maximum count. If a filename is supplied, information
// about the most recent checkins for that file only are returned. Specify a filename
// by adding ?serverPath=<serverPath>. Note that the filename must be expressed as a server
// pathname without the leading $/.
//
// E.g., to see the activity for a file $/teamProjectA/myFile, add this to the Url
// ?serverPath=teamProjectA/myfile
//
// This page returns no items if the file does not exist.

// ***************************************************
// (Default: on) Set this to true to throttle user requests by host/username
// ***************************************************
bool throttleRequests = true;

// Ignore requests from the same machine within this interval
int pesterInterval = 30 * 1000; // expressed in milliseconds

int maxSubjectLength = 60;

if (throttleRequests)
{
    // If the originating system has been serviced recently, drop the request
    object lastRequest = Context.Cache[User.Identity.Name+Request.UserHostAddress];
    if (lastRequest != null)
    {
        if ((int) lastRequest + pesterInterval < Environment.TickCount)
        {
            return;
        }
    }
}

// Default -- everything under $/
string serverPathname = VersionControlPath.RootFolder;
string username = null;

try
{
    // Check whether information on a specific file or userwas requested
    string pathname = Request.Params["serverPath"];
    if (pathname != null)
    {
        // The pathname does not contain a leading '$'
        serverPathname = VersionControlPath.Combine(VersionControlPath.RootFolder, pathname);
        // The pathname is valid.
    }
    username = Request.Params["user"];
}
catch
{
    Response.StatusCode = 404;
    Response.End();
}

// The maximum number of changes to return per query
int maxCheckinCount = 50;

string rssVersion = "2.0";
string rssTtl = "5";
string rssLanguage = "en-US";
string rssLink = Request.Url.ToString();
string rssEmptyComment = "None";
string rssTitle = "Team Foundation RSS Feed for Source Code Control Checkins";
string rssGenerator = "Sample RSS Feed Generator for Team Foundation";
string rssDescription = "<p>This feed provides information on Team Foundation Checkins. Each Team Foundation checkin is manifested as a changeset. The changesets that have been created recently are listed here.</p><p>This feed contains the most recent <i>{0}</i> checkin(s).</p>";
string rssItemOnBehalfOf = "(on behalf of {0}) ";
string rssItemTitle = "Check-in {0}: {1}";
string rssItemSubject = "Check-in contains changes to {0} item(s)";
string rssNoItemsAvailableDescription = "An error occurred obtaining the latest checkin information from Team Foundation. Try again later.";
string rssItemDescription = "<p><a href=\"{5}\">Changeset {0}</a> was checked in by <i>{1} {2}</i>on {3}. This checkin includes changes to {4} item(s).</p><p>Checkin comment: {6}</p><p>You can view the details of the checkin by selecting the provided link.</p>";
string exceptionMessage = "<p>An exception occurred while getting updated information from Team Foundation; the detailed exception message is <i>{0}</i></p>";
string itemNotFoundErrorMessage = "<p>There were no items found.</p>";
string accessDeniedErrorMessage = "<p>You do not have permission to obtain the Team Foundation checkin information.</p>";
string otherErrorMessage = "<p>An error occurred. This may be a transient error or a permanent one. Please check the event log for messages from VSTF Source Code Control</p>";

Response.ContentType = "text/xml";
XmlTextWriter xt = new XmlTextWriter(Response.OutputStream, null);
// Begin creating the XML document
xt.WriteStartElement("rss");
xt.WriteAttributeString("version", rssVersion);
xt.WriteStartElement("channel");
xt.WriteElementString("title", rssTitle);
xt.WriteElementString("ttl", rssTtl);
xt.WriteElementString("link", rssLink);
xt.WriteElementString("pubDate", DateTime.Now.ToString());
xt.WriteElementString("language", rssLanguage);
xt.WriteElementString("generator", rssGenerator);
try
{
    RecursionType recursionType = RecursionType.Full;

    // Obtain the list of changes from the mid-tier; the number of changes is reported in
    // the channel description.
    IEnumerable changesetEnum = null;
    try
    {
        TeamFoundationServer Tfs = null;
        object cacheEntry = Context.Cache["TeamFoundationServer"];
        if (cacheEntry == null)
        {
            // Note: only works on localhost
            Tfs = TeamFoundationServerFactory.GetServer("http://localhost:8080");
            Context.Cache["TeamFoundationServer"] = Tfs;
        }
        else
        {
            Tfs = (TeamFoundationServer) cacheEntry;
        }
        VersionControlServer Vcs = (VersionControlServer) Tfs.GetService(typeof(VersionControlServer));
        // Return changes
        changesetEnum = Vcs.QueryHistory(serverPathname,  // on this item
                                         VersionSpec.Latest, // item version
                                         0,               // that are not deleted
                                         recursionType,   // at or below this item
                                         username,         // user
                                         null,            // start version
                                         null,            // stop version
                                         maxCheckinCount, // Up to this many changes
                                         true,            // include changes
                                         false);

        int changeCount = 0;
        foreach (Changeset change in changesetEnum)
        {
            changeCount++;
        }
        xt.WriteElementString("description", String.Format(rssDescription, changeCount));
    }
    catch (Exception e)
    {
        if (e is ItemNotFoundException)
        {
            rssNoItemsAvailableDescription += itemNotFoundErrorMessage;
        }
        else
        {
            rssNoItemsAvailableDescription += otherErrorMessage;
        }
        rssNoItemsAvailableDescription += String.Format(exceptionMessage, e.Message);
        xt.WriteElementString("description", rssNoItemsAvailableDescription);
    }

    xt.WriteEndElement(); // channel

    // Create an item for each returned changeset.
    foreach (Changeset change in changesetEnum)
    {
        xt.WriteStartElement("item");
        string onBehalfOf = null;

        // Include the committer if it differs from the changeset owner.
        // This occurs when a proxy agent performs the checkin.
        if (!change.Owner.Equals(change.Committer))
        {
            onBehalfOf = String.Format(rssItemOnBehalfOf, change.Owner);
        }

        // Generate a subject based on the checkin comment
        string subject = rssEmptyComment;
        if (!String.IsNullOrEmpty(change.Comment))
        {
            subject = change.Comment.Trim();
        }
        int max = Math.Min(subject.Length, maxSubjectLength);
        int newline = subject.IndexOf('\n', 0, max);
        if (newline != -1)
        {
            if (newline > 0 && subject[newline - 1] == '\r')
            {
                newline--;
            }
            subject = subject.Substring(0, newline);
        }
        if (subject.Length >= maxSubjectLength)
        {
            subject = String.Concat(subject.Substring(0, maxSubjectLength), "...");
        }

        xt.WriteElementString("title", String.Format(rssItemTitle, change.ChangesetId, subject));

        string csLink = HttpUtility.HtmlEncode(new ChangesetUri(
                                                     String.Format("{0}://{1}:{2}",
                                                                  Request.Url.Scheme,
                                                                  Request.Url.Host,
                                                                  Request.Url.Port),
                                                     change.ChangesetId,
                                                     UriType.Extended).ToUrl());

        xt.WriteElementString("description", String.Format(rssItemDescription,
                                                           change.ChangesetId,
                                                           change.Committer,
                                                           onBehalfOf,
                                                           change.CreationDate,
                                                           change.Changes.Length,
                                                           csLink,
                                                           change.Comment));
        xt.WriteElementString("link", csLink);

        xt.WriteElementString("subject", String.Format(rssItemSubject,change.Changes.Length));
        xt.WriteElementString("author", onBehalfOf == null ? change.Committer : change.Owner);
        xt.WriteElementString("pubDate", change.CreationDate.ToString());
        xt.WriteElementString("guid", change.ChangesetId.ToString());
        xt.WriteEndElement(); // item
    }
}
catch (Exception e)
{
    Response.StatusCode = 404;
    Response.End();
}
finally
{
    xt.Close();
    if (throttleRequests)
    {
        Context.Cache[Request.UserHostAddress] = Environment.TickCount;
    }
}
%>

Team Foundation: What's my server doing? (Part 2)

In my last post, I wrote about the mechanism within Team Foundation that records web method activity. You can mine the database to and learn much about how Team Foundation is being used. Recall that the web method information is written to a database so the full power of SQL queries is available.

List all Team Foundation activity

Combine the command and parameter table (the left join is necessary since some web method rows in the command table may not record parameter information -- more on this later)

select *
from tbl_command c
left join tbl_parameter p
on c.commandid = p.commandid

List all Team Foundation Version Control activity

Same as the previous query with the addition of filtering on the Application column (command table).

select *
from tbl_command c
left join tbl_parameter p
on c.commandid = p.commandid
-- Just Version Control (other choices: Data Warehouse, Integration, Work Item Tracking, Proxy)
where c.Application = 'Version Control'

Show the web methods with the longest execution time

The ExecutionTime column contains the execution time, in microseconds, of the web method.

select *
from tbl_command c
left join tbl_parameter p
on c.commandid = p.commandid
order by c.ExecutionTime desc

Show the web methods called the most

select command, count (command) as TimesCalled
from tbl_command c
group by command
order by TimesCalled desc

Show the web method calls made by userN

The IdentityName column contains the Windows account name of the caller.

select *
from tbl_command c
left join tbl_parameter p
on c.commandid = p.commandid
-- IdentityName contains the account name of the user (domain\account)
where c.IdentityName like '%userN%'

Show the web method calls made from Visual Studio

The UserAgent column contains the name of the executable from which the web method call originates.

select *
from tbl_command c
left join tbl_parameter p
on c.commandid = p.commandid
-- UserAgent contains the executable name
where c.Useragent like '%devenv.exe%'

Other tidbits

• Rows in the parameter table contain the commandId value of the corresponding row in the command table. Note that there may be multiple rows in the parameter table for a single command
• Parameters are recorded when (1) An error occurs during web method execution (indicated by the Status column of the command table set to -1); (2) The web method took longer than 30 seconds to execute (recall that the ExecutionTime values are in microseconds) or (3) the Web method logging level is set to All in the web.config file.

Team Foundation: What's my server doing?

We designed a mechanism to record web method activity for all of the Team Foundation web applications. Information about each web method invocation is recorded in a single database.

We don't provide any ways to get to that data -- however, if you have access to the data tier you can write a few SQL queries to see what your Team Foundation web applications are doing.

Web method logging is controlled by a setting in each web application's web.config file. The default configuration disables web method logging but it's pretty simple to enable it.

Locate each web.config file under the installation directory, i.e., D:\Program Files\Microsoft Visual Studio 2005 Enterprise Server.

In the <appSettings> section you should see something like the following:

     <!--  WEB METHOD LOGGING
          Specify whether web methods are to be logged always, never, or only
          when errors occurs. Without a value for commandLogging, web methods
          are not logged.  Valid values are:
              None     (web methods are never logged)
              Normal   (log everything, except certain high-frequency web methods)
              All      (log everything, even certain high-frequency web methods)
              OnError  (only log web methods that have errors)

              Values specified here take precedence over values set administratively via
              web method.
        -->
        <add key="commandLogging" value="None"/>

Change None to Normal or All and web method activity will now be recorded.

I'll go into the details of how we collect and record the data if enough people are interested. Note that the data is spooled for a short while before it's written to the database.

The database (for now, it's will be renamed) is called VSTEAMSCCAdmin. There are some historical reasons for including SCC in the name; it'll be called TFSActivityLogging in the shipping product.

There are two tables in this database: one records information about each web method and the other contains parameter information for each web method.

A simple query that shows web method activity plus the parameters (SOAP parameters) is (note: the LEFT join is necessary since there may not be values from the parameter table for each command):

select *
from tbl_command (nolock) c
left join tbl_parameter  (nolock) p
on c.commandid = p.commandid

The command table contains information about the web method itself and includes:

• Command identifier A unique id used as a key in the parameter table
• Application Identifies the Team Foundation Web Application servicing the web method
• Command The name of the web method
• StartTime When the web method was invoked
• ExecutionTime How long the web method executed (microseconds)
• IdentityName The user invoking the web method
• IPAddress Origination of the web method execution request
• UserAgent The provided user agent string (this will contain the name of the executable as well)
• UniqueIdentifier (optional) When specified, this value correlates multiple web method records. E.g., a command like checkin will perform multiple web method calls. The identifier correlates the web method calls made for a specific occurrence of the command.
• CommandIdentifier (optional) The name of the command-line tool that's invoking the web method (note that a single command like shelve will perform multiple web methods)

The parameter table is pretty straight forward -- note that parameters are conditionally reported.

• CommandId Correlates to an entry in the command table
• ParameterName The name of the parameter
• ParameterValue The value of the parameter

Version Control RSS Feed

Team Foundation Watson Reports: We're listening

We've been collecting Watson reports with the Beta 2 release. As of this week, we've created 81 bugs from the Watson reports that we've received from Beta2 and internal sites. All but 2 of the bugs have been resolved.

Please keep those Watson reports coming to us -- we actively review, prioritize, and follow up on them.

-jeff

Sample: Version Control RSS Feed

<%@ Page Language="c#" %>
<%@ OutputCache Duration="20" Location="Server" VaryByParam="state" VaryByCustom="minorversion" VaryByHeader="Accept-Language"%>

<%
// "Copyright © Microsoft Corporation.  All rights reserved.  These Samples are based in part on the Extensible Markup Language (XML) 1.0 (Third Edition) specification Copyright © 2004 W3C® (MIT, ERCIM, Keio. All rights reserved. http://www.w3.org/consortium/legal/2002/copyright-documents-20021231."
%>

<%
//This posting is provided "AS IS" with no warranties of any kind and confers no rights.  Use of samples included in this posting is subject to the terms specified at http://www.microsoft.com/info/cpyright.htm.
%>

<%@ Import Namespace="System" %>
<%@ Import Namespace="System.Collections" %>
<%@ Import Namespace="System.IO" %>
<%@ Import Namespace="System.Web" %>
<%@ Import Namespace="System.Xml" %>
<%@ Import Namespace="Microsoft.TeamFoundation.Client" %>
<%@ Import Namespace="Microsoft.TeamFoundation.VersionControl.Client"%>
<%@ Import Namespace="Microsoft.TeamFoundation.VersionControl.Common" %>

<%
// Generate an RSS feed for Team Foundation Source Code Control checkins
//
// Note: Only one request per "pester interval" from each system will be honored. See
//       pesterInterval below.
//
// This feed returns information about the most recent N checkins. See maxCheckinCount
// below.
//
// Invoking this page without any parameters returns information about all Team
// Foundation checkins up to the maximum count. If a filename is supplied, information
// about the most recent checkins for that file only are returned. Specify a filename
// by adding ?serverPath=<serverPath>. Note that the filename must be expressed as a server
// pathname without the leading $/.
//
// E.g., to see the activity for a file $/teamProjectA/myFile, add this to the Url
// ?serverPath=teamProjectA/myfile
//
// This page returns no items if the file does not exist.

// ***************************************************
// (Default: off) Set this to true to throttle user requests by host/username
// ***************************************************
bool throttleRequests = false;

// Ignore requests from the same machine within this interval
int pesterInterval = 30 * 1000; // expressed in milliseconds

if (throttleRequests)
{
    // If the originating system has been serviced recently, drop the request
    object lastRequest = Context.Cache[User.Identity.Name+Request.UserHostAddress];
    if (lastRequest != null)
    {
        if ((int) lastRequest + pesterInterval < Environment.TickCount)
        {
            return;
        }
    }
}

// Default -- everything under $/
string serverPathname = VersionControlPath.RootFolder;

try
{
    // Check whether information on a specific file was requested
    string pathname = Request.Params["serverPath"];
    if (pathname != null)
    {
        // The pathname does not contain a leading '$'
        serverPathname = VersionControlPath.Combine(VersionControlPath.RootFolder, pathname);
        // The pathname is valid.
    }
}
catch
{
    Response.StatusCode = 404;
    Response.End();
}

// The maximum number of changes to return per query
int maxCheckinCount = 50;

string rssVersion = "2.0";
string rssTtl = "5";
string rssLanguage = "en-US";
string rssLink = Request.Url.ToString();
string rssTitle = "Team Foundation RSS Feed for Source Code Control Checkins";
string rssGenerator = "Sample RSS Feed Generator for Team Foundation";
string rssDescription = "<p>This feed provides information on Team Foundation Checkins. Each Team Foundation checkin is manifested as a changeset. The changesets that have been created recently are listed here.</p><p>This feed contains the most recent <i>{0}</i> checkin(s).</p>";
string rssItemOnBehalfOf = "(on behalf of {0}) ";
string rssItemTitle = "Checkin {0} [{1} items]";
string rssNoItemsAvailableDescription = "An error occurred obtaining the latest checkin information from Team Foundation. Try again later.";
string rssItemDescription = "<p><a href=\"{5}\">Changeset {0}</a> was checked in by <i>{1} {2}</i>on {3}. This checkin includes changes to {4} item(s).</p><p>You can view the details of the checkin by selecting the provided link.</p>.";
string exceptionMessage = "<p>An exception occurred while getting updated information from Team Foundation; the detailed exception message is <i>{0}</i></p>";
string itemNotFoundErrorMessage = "<p>There were no items found.</p>";
string accessDeniedErrorMessage = "<p>You do not have permission to obtain the Team Foundation checkin information.</p>";
string otherErrorMessage = "<p>An error occurred. This may be a transient error or a permanent one. Please check the event log for messages from VSTF Source Code Control</p>";

Response.ContentType = "text/xml";
XmlTextWriter xt = new XmlTextWriter(Response.OutputStream, null);
// Begin creating the XML document
xt.WriteStartElement("rss");
xt.WriteAttributeString("version", rssVersion);
xt.WriteStartElement("channel");
xt.WriteElementString("title", rssTitle);
xt.WriteElementString("ttl", rssTtl);
xt.WriteElementString("link", rssLink);
xt.WriteElementString("pubDate", DateTime.Now.ToString());
xt.WriteElementString("language", rssLanguage);
xt.WriteElementString("generator", rssGenerator);
try
{
    RecursionType recursionType = RecursionType.Full;

    // Obtain the list of changes from the mid-tier; the number of changes is reported in
    // the channel description.
    IEnumerable changesetEnum = null;
    try
    {
        TeamFoundationServer Tfs = null;
        object cacheEntry = Context.Cache["TeamFoundationServer"];
        if (cacheEntry == null)
        {
            // Note: only works on localhost
            Tfs = new TeamFoundationServer("http://localhost:8080");
            Context.Cache["TeamFoundationServer"] = Tfs;
        }
        else
        {
            Tfs = (TeamFoundationServer) cacheEntry;
        }
        VersionControlServer Vcs = (VersionControlServer) Tfs.GetService(typeof(VersionControlServer));
        // Return changes
        changesetEnum = Vcs.QueryHistory(serverPathname,  // on this item
                                         VersionSpec.Latest, // item version
                                         0,               // that are not deleted
                                         recursionType,   // at or below this item
                                         null,            // user
                                         null,            // start version
                                         null,            // stop version
                                         maxCheckinCount, // Up to this many changes
                                         true,            // include changes
                                         false);

        int changeCount = 0;
        foreach (Changeset change in changesetEnum)
        {
            changeCount++;
        }
        xt.WriteElementString("description", String.Format(rssDescription, changeCount));
    }
    catch (Exception e)
    {
        if (e is ItemNotFoundException)
        {
            rssNoItemsAvailableDescription += itemNotFoundErrorMessage;
        }
        else
        {
            rssNoItemsAvailableDescription += otherErrorMessage;
        }
        rssNoItemsAvailableDescription += String.Format(exceptionMessage, e.Message);
        xt.WriteElementString("description", rssNoItemsAvailableDescription);
    }

    xt.WriteEndElement(); // channel

    // Create an item for each returned changeset.
    foreach (Changeset change in changesetEnum)
    {
        xt.WriteStartElement("item");
        string onBehalfOf = null;

        // Include the committer if it differs from the changeset owner.
        // This occurs when a proxy agent performs the checkin.
        if (!change.Owner.Equals(change.Committer))
        {
            onBehalfOf = String.Format(rssItemOnBehalfOf, change.Owner);
        }

        xt.WriteElementString("title", String.Format(rssItemTitle,
                                                     change.ChangesetId,
                                                     change.Changes.Length));

        string csLink = HttpUtility.HtmlEncode(new ChangesetUri(
                                                     String.Format("{0}://{1}:{2}",
                                                                  Request.Url.Scheme,
                                                                  Request.Url.Host,
                                                                  Request.Url.Port),
                                                     change.ChangesetId,
                                                     UriType.Extended).ToUrl());

        xt.WriteElementString("description", String.Format(rssItemDescription,
                                                           change.ChangesetId,
                                                           change.Committer,
                                                           onBehalfOf,
                                                           change.CreationDate,
                                                           change.Changes.Length,
                                                           csLink));
        xt.WriteElementString("link", csLink);
        xt.WriteElementString("author", onBehalfOf == null ? change.Owner : change.Committer);
        xt.WriteElementString("pubDate", change.CreationDate.ToString());
        xt.WriteElementString("guid", change.ChangesetId.ToString());
        xt.WriteEndElement(); // item
    }
}
catch (Exception e)
{
    Response.StatusCode = 404;
    Response.End();
}
finally
{
    xt.Close();
    if (throttleRequests)
    {
        Context.Cache[Request.UserHostAddress] = Environment.TickCount;
    }
}
%>

Part 3: What's my source control server doing? A RSS feed to keep track of checkins

I haven't forgotten about the RSS feed generator. There were some issues that I needed to resolve with the sample code before it can be released; this required a few changes -- these include:

• Instead of using public methods in the Version Control server assemblies, the Version Control Client Object Model (OM) is used instead. The OM supports a rich API and is the intended, suggested, and supported way by which applications interact with Team Foundation.
• You should note that the RSS feed generator is a sample and is provided "AS IS". The file contains the disclaimers and copyright notice.

Caveats

• This posting is provided "AS IS" with no warranties of any kind and confers no rights.  Use of samples included in this posting is subject to the terms specified at http://www.microsoft.com/info/cpyright.htm.
• It may not work with the RSS reader that you're using (I've been using SharpReader).
• It may not work with your environment.
• I have verified that it will work with our upcoming CTP. Changes made to Team Foundation subsequent may be incompatible with the RSS feed generator.
• The RSS feed generator uses the credentials of the client invoking it to obtain information about checkins (read access to at least one item in the changeset is required to obtain information about the changeset).

RSS Feed Generator Features

• User throttle: Within a 30-second interval, only the first request from each user is serviced (duplicate requests are ignored). You can disable this behavior by setting throttleRequests to false. The interval is determined by pesterInterval.
• Path-specific subscriptions: You can include a server path (e.g., $/teamProject/somePathOfInterest) to subscribe to only the area(s) of interest. If server path specifies a folder, than any changes at or below that folder are included in the feed. If server path specifies a file, only changes to that file are in the feed. The default server path is the folder $/
• Security: The caller's credentials are used to obtain information about checkin activity. The links returned in the feed are for changeset webviews.

Example

Each checkin is represented by an individual item in the feed; e.g., this is information about the 3rd checkin:

Changeset 3 was checked in by DOMAIN\user on 7/21/2005 12:56:05 PM. This checkin includes changes to 15 item(s).

You can view the details of the checkin by selecting the provided link.

Selecting the changeset link will display the webview of that changeset:

Check-in 3: fsdfsadf

Summary

None

None

None

fsdfsadf

Resolved Work Items

Items

Notes:
- All dates and times are shown in Eastern Daylight Time (GMT -04:00:00).
Provided by Microsoft Visual Studio® 2005 Team System

Installation

The RSS feed generator is a .ASPX file so all you need to do is copy the file under the Version Control subdirectory and then subscribe to the feed. The following example assumes that Team Foundation is using port 8080 (default). 'SCC' is the name of the Version Control web application (you can view the web site layout using inetmgr).

To install and subscribe to the feedFor example,

   copy CheckinRssFeed.aspx %ProgramFiles%\Microsoft Visual Studio 2005 Enterprise Server\HATIISDIR

Using your RSS reader, subscribe to http://machine:8080/SCC/CheckinRssFeed.aspx

Why we want Watson reports

One of the features we added to the mid-tier web services was the ability to create and file reports when error and/or unexpected conditions occur. When these happen, a information about the condition is gathered and then a report is filed with the Watson reporting mechanism.

The next time an administrator logs in to the mid-tier machine, the Watson reporting UI should appear asking if you want to send the queued reports. Please consider doing so -- we really need this information to see what kind of real-world problems are occurring. We include information about the error condition and stack trace of the thread active when the condition occurred in the report.

You might wonder what happens to the Watson reports that are sent to Microsoft. Each week, the Team Foundation Watson Review team (a representative from the test organization and myself) review the Watson report that arrived in the last week. We focus on the newly arriving Watson reports and the reports that have been reported the most. We used to investigate only the top 10 reports (as measured by our internal Watson tool) but have now extended our focus to the top 60 or so (we look at any report that has been reported more than a few times).

We look at reports from Beta 2 deployments and some other key deployments or test beds that are internal to Microsoft.

After we investigate the set of Watson reports, we send a summarization that includes the Watson report, status, the bug (if any -- many problems have already been fixed) details, and the developer responsible for fixing it. This summarization gets wide visibility by design -- we even send it to our PUM's boss. I'm happy  to report that we've been making great progress in fixing issues illustrated by the Watson process.

One more thing -- and I can't stress this enough -- we really do look at the Watson reports that we receive and make every attempt to correct the issue(s) that are contained in the report. Please continue to allow the reports to be sent.

Part 2: What's my source control server doing? A RSS feed to keep track of checkins

In my last post, I started describing an RSS feed generator for Team Foundation Source Control. Note that this is an experimental endeavor and not supported in any way by Microsoft as it is not part of the Team Foundation product.

I described, in brief, the features that it supports and touched on security. I won't repeat that information here.

Feed Contents

The RSS feed provides information about recent checkins -- there is one RSS item for each checkin that has occurred. Each item contains the following

• The date, time, and number of changes in the checkin.
• Who performed the checkin
• A link to a changeset webview.

The contents of the item include meta information about the checkin but not the checkin details (note the security information in my last post) and a link to the changeset webview. I'll talk about what those are in a minute.

By default, i.e., if a file path is not supplied, information about all checkins is returned. When a file path is part of the Url, i.e.,g url?filename=filepath, information about only the filepath is returned. If it refers to a directory, information about checkins at or below the directory is returned; otherwise, information about checkins for that file only are returned.

Changeset Webview

Webviews are a core Team Foundation feature that provide web-based access to Team Foundation artifacts. Each Team Foundation component has a set of artifacts. E.g., Workitem Tracking has an artifact for a work item (no surprise) and Source Control has artifacts for changesets and files.

The changeset webview shows the changeset, e.g., who created the changeset, when it was created, what changes are included, etc. In Beta2, the changeset webview appearance is crude but contains all of the information from a changeset. We've polished the appearance and included extra information in the bits after Beta2.

RSS Feed Generator

The RSS feed provides information about checkins; for each checkin, a changeset webview link is provided. I've mentioned what a changeset webview contains so its now time to discuss the RSS feed. Once installed, you can subscribe to the RSS feed by specifying the .aspx page that generates the feed, e.g.,

http://server:port/applicationName/application/CheckinRssFeed.aspx

The RSS feed generator uses the following .NET and Team Foundation assemblies:

• System (System, System.Web, System.XML namespaces)
• Microsoft.VisualStudio.Hatteras.Bis
• Microsoft.VisualStudio.Hatteras.Util
• Microsoft.VisualStudio.Hatteras.Server

Keep in mind that assembly names and namespaces will be changing for RTM. For Beta2, project names (e.g., Hatteras, Currituck) and temporary names (Bis) are still used. Assembly names, namespaces, classes, methods and other places where internal project names are in use will be different.

In my next post, I'll start posting snippets of the feed.

In the meantime, let me know if there are specific things you'd like to see.

What's my source control server doing? A RSS feed to keep track of checkins

Team Foundation RSS Feed Generator for Source Control

Some time ago, I switched to a much more efficient way of keeping up with news and other events. Instead of visiting several web sites to keep up with what's happening, I started using a web-based blog aggregator. There are many out there, the one what worked the best for me and my situation is http://www.bloglines.com. This way, I could keep track of what's going on by visiting one web site and viewing the news articles that I wanted to see. Of course, if one of the sites doesn't have an RSS feed, than I was faced with either ditching that site or visiting the site. Usually, I ended up ditching the site in favor of viewing articles using the RSS reader.

In that spirit, I started to look into creating an RSS feed for Team Foundation. I work on the Source Control part of Team Foundation, so naturally, I started there. Before I get into what I actually did to create the RSS feed, let me explain a little about Team Foundation Alerts. An alert is an out-of-band notification that some condition that you expressed interest in has occurred. When the condition occurs, Team Foundation will send an email with the details. We have alerts that notify you when

1. A work item is assigned to you (or a work item that is assigned to you changes)
2. A checkin to has occurred.
3. A build has started
4. A build has completed

Alerts are created by subscribing  to one of these events and are scoped to a Team Project. When you subscribe to an alert (or more commonly, the Team Project administrator has created a subscription for you), you'll only receive alerts that related to that Team Project. You have to use Visual Studio to subscribe to an alert. Select a Team Project, then use Team -> Alerts. Select the event(s) and supply an email address.

Back to RSS feeds. I receive a lot of e-mail each day and decided that I'd rather find out about checkins that have occurred differently. Thus, the RSS feed generator for Team Project Source Control was born.

The RSS feed is implemented in a .ASPX file and makes use of public interfaces the Team Foundation source control assemblies to retrieve information about the most recent checkins. I'm not going to explain the details of what constitutes an RSS feed; there are many websites that do this. Instead, I'll focus on on this feed works.

Features

• Returns information about the most recent checkins to Team Foundation
• If a pathname is supplied, checkins relating to that path or below are returned
• Only services 1 request per pester-interval for mal-behaved RSS aggregators
• Only uses public interfaces in the Team Foundation assemblies

Overview

At a high level, this is what the RSS feed generator I wrote does:

• Decide whether to process the request. If the sender was serviced within the pester interval, the request is ignored.
• Check whether the request contains a filename. When a filename is specified, checkins to that filename (or, if it's a directory, files in the directory tree) is returned. When a filename is not specified, information about all recent checkins is returned.
• Gather information about the checkins and prepare the RSS XML stream.

Sounds pretty simple, right? There are a few details buried in the .aspx file and you might be wondering about security. Team Foundation Source Control has a security model that provides authorization and information protection and has safeguards against to prevent information disclosure.

Security

A checkin represents a set of file changes. In order for a user to "see" the checkin, the user must have access to at least one element of the checkin. When a user doesn't have access any elements of the checkin, the checkin cannot be viewed by that user.

Therefore, the RSS aggregator used to obtain the feeds, must have read access to the Source Control repository. The repository provides a hierarchical view of the files that it contains rooted at $/. Team Projects are created as the top most elements, e.g., a repository with Team Projects Project1 and Project2 are represented this way:

   $/Project1/

         Subdirectory1

         Subdirectory2

   $/Project2/

         OtherSubdirectory1

         OtherSubdirectory2

Access to the files and directories is governed by a set of inheritable Access Control List (ACL) entries. Each ACL specifies which identities (users and/or groups) are granted or denied access to elements. You can view the permission settings of an item from the command line using the Source Control command line tool h.exe (to be renamed when we ship) as follows

   h perm $/Project1

   ===============================================================================
   Server item: $/Project1 (Inherit: Yes)
     Identity: Namespace Administrators
       Allow:
       Deny:
       Allow (Inherited): Read, PendChange, Checkin, Label, Lock, ReviseOther,
                          UnlockOther, UndoOther, LabelOther, AdminProjectRights, CheckinOther
       Deny (Inherited):

     Identity: Service Accounts
       Allow:
       Deny:
       Allow (Inherited): Read, PendChange, Checkin, Label, Lock, ReviseOther,
                          UnlockOther, UndoOther, LabelOther, AdminProjectRights, CheckinOther
       Deny (Inherited):

This shows that there are no explicit permissions assigned to $/Project1 since the Allow: and Deny: entries are empty. There are inherited permissions, however. This example also shows the default permissions that exist when Team Project is installed.

NOTE: Back to the RSS feed aggregator: the RSS aggregator must have read permission to the (sub)-tree of the repository that it gathers information from. You can use either the Permissions dialog within Visual Studio or, if you're a gear-head like me, use the h perm command to view and change the permission settings.

What do I really mean? The identity of the process that requests the feed must have read access.

You're probably wondering about the huge security hole that exists here, right? You get extra points for thinking about this but I've already taken care of it. The potential security hole is this:

1. An RSS aggregator gathers checkin information for a site.
2. Users connect to the RSS aggregator to get checkin information
3. At least one of the users in step 2 can now see checkin information that, had they requested it directly using h changeset someNumber would be denied.

The RSS feed returns hyperlinks that, when followed, are processed according to the security associated with the identity opening the link. The RSS feed returns meta information about the checkin and the user uses the meta information (i.e., the hyperlink), to actually see the details of the changeset.

Whew! That's all for now, I'll write more about the details of the RSS feed in an upcoming blog.

jeff

Team Foundation Alerts

Team Foundation alerts are a way for users to receive notifications when certain conditions occur. You can subscribe to these alerts on your client system through Team -> Alerts.

Team Foundation sends alerts for these events -- each of these events are scoped to the active Team Project. Note that if you have subscriptions to multiple team projects and one of the following events includes items from multiple team projects, than you will only receive one e-mail notification.

1. When a work item assigned to you is changed (created, modified)
2. When a checkin to the current Team Project occurs
3. When a build completes.
4. When build status changes.

Where do Alerts Come From?

Alerts are generated following one of the activities listed in the alert subscription tab. The Team Foundation component that owns the activity, prepares an XML document and sends that to the Team Foundation Core Services component (aka "BIS") since that component owns the subscription data. When BIS matches an XML document (this is really an "event") with one or more subscribers, it begins the process of preparing an email for each of the subscriptions. This process includes content scrubbing by the Team Foundation component that owns the "event." Generally speaking, the event contains all of the information that is associated with the activity. E.g, a "checkin" event contains all of the changes that were made as part of that checkin. The list of changes is scrubbed or filtered so the email contains only those items to which the subscriber has read access. In other words, the email will only contain those items to which the subscriber could view directly.  If the subscriber lacks read access to a file that was checked in, the file is not included in the email alert.

After the event has been scrubbed, BIS takes over and prepares the actual email by invoking the XSL transform associated with the event. After the transformation, the email is then sent to the email address associated with the subscription.

This process repeats for each subscriber. You don't need to create a subscription for each user; the email address can be for a group of users.

Alert Appearance

Alerts are created from an XML document containing the alert information and an XSL transform. The XSL transforms are located on the app-tier in BISIISDIR\bisserver\events\eventschemas. The schemas for the XML documents are currently in one of two places:

1. Team Foundation registration data
2. The same directory as the XSL transforms

Use the inetmgr to view the Team Foundation registration data:

1. Navigate to VSTFWeb and expand bisserver.
2. RMB on registration.asmx and select 'Browse..."  This will start IE navigated to this page.
3. Select the GetRegistrationEntries method.
4. After the page loads, press invoke
5. You'll see the registration information for the services. As of CTP, not all services were registering their schemas.

The XSL transform directly controls the look and feel of the alert. We've put a lot of effort into the upcoming Beta2 alert appearance and they should be very close to what you'll see in the shipping product. The CTP alerts contain most of the information but are inconsistent and certainly not polished.

You can change the appearance of the alerts by modifying the transform that creates the alert.  If you make changes to the transform that introduce XSL processing errors (if you don't know what these are, you might not want to make any changes to the transforms!) you will have prevented the alert from being sent. Be careful.

I'll discuss alerts and web views in an upcoming blog.

Why BisRegEdit is sometimes the reported culprit of a failed install.

I've been reading the newsgroup postings setup for Team System. Of particular interest, are those dealing with setup issues as I've spent a fair amount of time working with "setup" to make sure that the pieces work together.

There are two types of installs that we do around here.

1. Scripted installs using a combination of batch, vbasic, and other "scripts".
2. Layout install which installs from CABs, uses setup.exe.

I've done a fair amount of work with the first type and am familiar with many of the steps necessary. BTW, we try very hard to make sure that the product of either install are equivalent.

Ok, so how does BisRegEdit fit into the setup process? "BIS" (will be renamed when we RTM), provides infrastructure services. One of those functions is to help the multiple services (Work Item Tracking, Build, Source Code Control, Data Warehouse, etc) locate each other. In this case, it acts as a clearing house. The information that it provides comes from each service registering information with BIS at install time. There are many ways to provide this information. The Source Code Control team chose to provide the registration information through a .ASPX page (dynamically). What this means, which may be subtle, is that the Source Code Control system which was just installed is invoked to provide the data. Thus, if part of the installation has failed, than this step usually (but not always) will highlight a failure elsewhere in the installation process.

OK, I said this would be about why BisRegedit is often listed (wrongly) as the culprit. Let's look at the actual invocation line and pick it apart:
"C:\Program Files\Microsoft Visual Studio 2005 Enterprise Server\BISIISDIR\sdk\bin\bisregedit.exe"  "http://TEAMSYS:8080/SCC/public/application/ServiceDefinition.aspx" TEAMSYS.BURTON.COM BisDB

Bisregedit accepts command lines of the form:  <xml-document> <TeamSystem Database Server> <BisDatabase>
where

• xml-document contains the registration data for the service.
• TeamSystem Database Server is the name of the data-tier system
• BisDatabase is the name of the database where the data will be stored.

In the case of the Source Code Control system, the registration data is obtained by invoking the Source Code Control system (just installed!). The page at <InstallDir>/SCC/public/application/ServiceDefinition.aspx provides this information. From a browser (on an installation that's working), you can navigate to the same Url to see the data. E.g., open your favorite browser and enter http://TEAMSYS:8080/SCC/public/application/ServiceDefinition.aspx

When everything is installed properly, the BisRegEdit loads the contents of the document using the following pattern:
    XmlDocument registrationDoc = new XmlDocument();
 registrationDocument.Load(string xmlDocument); // in this example, the parameter is the Url listed above but a file can be supplied

When BisRegedit fails, it may be due to a few things; I usually triage failures as follows:

1. Installation log: check for failures earlier in the process. The installation log is on the client system.
2. Event log of mid tier: check for error event log entries (the source will be VSTF * or TFS) in the application log.
3. Make sure that ServiceDefinition.aspx permits anonymous access.
4. Check ASP.NET version  for TF web site (should be 2.x)
5. IIS log for web site looking for 5xx or 403 status codes (generally, 401s are ok if they're followed by a 200).

If there aren't any errors in the install log that are obvious, then the problem may be isolated to the Source Code Control system. I always check the event log first (on the mid/app-tier).  Look for events in the application log with the source VSTF Source Code Control. If there are any, than supplying that data will help us determine what went wrong.

If there are no event log entries sourced by VSTF Source Code Control, check the system log for W3SVC entries around the same time. Even the ones marked information are important.

If there are no relevant event log entries, then make sure that the ServiceDefinition.aspx page permits anonymous access. It's supposed to be set during the installation. Use the inetmgr to view: Start, Run, Inetmgr (or start inetmgr from a command prompt). Open web sites, open VSTFWebSite, open SCC, open public, open application, RMB ServiceDefinition.aspx and look at properties, select the file security tab, select 'edit' on authentication (top-most item). Verify that the anonymous access is enabled.

If anonymous access is enabled for the page, check the properties of the web site using inetmgr.Open Web Sites and RMB on VSTFWebSite (I think that's the name). Check the
ASP.NET tab and make sure the ASP.NET version is 2.x or greater. If it's not 2.x, then the wrong version of ASP.NET is associated with the web site and Team Foundation will not work.

While you're looking at the properties, select the web site tab. Make sure that IIS logging is enabled, and select the 'Advanced' button. Note the log file directory at the bottom. This is the IIS log, if there are errors indicated here, than it's possible that the web service didn't receive the request from IIS. Normally, you'll see a pattern of calls with status 401, 401, then 200 (at the end of each line).  Open the log file corresponding to the date of the failed installation (note the format of each file). There should be a line with a 500 that corresponds to the 500 internal server error that you saw earlier. There might be a 403 as well. This information is important to share when posting information about a problem.

There are many of us involved with Team System that also read the newsgroups on a frequent basis. By sharing your post there, others can possibly benefit.

Hope that helps -- let me know if something isn't clear.

Introduction

Hi, my name is Jeff Lucovsky. I have been @ Microsoft since September 2003 working exclusively on Source Code Control. Most of my day-to-day work involves changes to the application tier but I have made some changes to the client-tier. Prior to Microsoft, I earned a living by working on that other operating system doing a wide variety of things including file systems, network file system (NFS), security, and TCP/IP. I also worked on an NT-based disk array processor for CLARiiON (now part of EMC), network mandatory access control for NT4.0 systems, intrusion detection systems (IDS) and most recently control software for Racemi (an Atlanta-based company).

Since I've joined Microsoft, I've worked on infrastructure -- administrative and other related functions mainly. This includes web method logging, event log usage, performance counters, e-mail notification (these are called Alerts), and application tier Watson interaction (part of Beta2). I've even dabbled with our help system on the client system.

I'll follow up with a post on some setup issues that folks are having with the recent CTP bits.