bharry's WebLog

Power Tools and Software Assurance

I love our Power Tool program.  I really enjoy the opportunity to deliver stuff that people are wanting every few months.  I also really like the very positive feedback we get on them.  I'm always looking for the next "killer" Power Tool feature that we can build.

One of the lingering questions that has never been answered and we mostly ignore is "How are Power Tools monetized?"  Right now, it kind of works like this...  Power Tools generally support the latest release of TFS.  So one of the benefits of buying the latest release is a new stream of Power Tools until the next TFS release.

The biggest lingering question is what about Software Assurance?  People who purchase Software Assurance are paying "extra" to make sure they get all the latest updates at no additional charge.  Many times, over the last few years, we've asked ourselves, "Should the right to use the Power Tools be tied to Software Assurance?".  From the customer's perspective, it would make them feel like they are getting even more for their money.  From our perspective, it would increase the likelihood customers would buy SA.

Why do I care if customers buy SA?  Well, of course I like the revenue but it's more than that.  We make decisions all of the time about how to release things, what to charge for them, etc.  When customers are on SA, it makes these decisions much easier.  We can simply release them to all licensed customers and not worrry about it.  Otherwise, we're left debating whether we should hold onto it for the next "big" release.  A great example is the features we put in service packs.  Every feature we put in there becomes one less feature in the next paid upgrade - meaning less stuff we can make a big deal about when the product ships and use to encourage people to upgrade.  However, if everyone is on SA, it's easier for us to accept because we can convince ourselves that people paid for it by having SA.

For these kinds of reasons, my goal it to have all of our customers on SA - then I don't need to spend any time worrying about how we are going to monetize things and I can spend all my time thinking about the next set of features we are going to get in your hands.

Anyway, there's no action right now.  However, I wanted to let you know this issue is on my mind.  The thing that brought it back to the top was a recent conversation with one of our sales people about a conversation with a customer who didn't want to buy SA and one of their reasons was that they get Power Tools without it and therefore they couldn't see sufficient value.  I'm going to continue to think about this issue and try to figure out how to have my cake and eat it too.  One idea that has flitted across my brain is releasing Power Tools to SA customers "first".  Perhaps we give SA customers access to Power Tools 3 or 6 months before non-SA customers.  I don't know...

If you have thoughts you'd like to share, I'm all ears.

Brian

July '08 DevDiv TFS Dogfood Statistics

I missed the June Dogfood statistics - sorry about that.  This report represents the change since my last dogfood report in May.  The big thing that you will observe is that downloads have dropped dramatically (from a peak around ~150,000,000 to ~50,000,000).  The reason for this is that we installed a TFS proxy on our corp net and had the majority of users configure their clients to use it.  The proxy is 2 machines configured behind an NLB load balancer.  The reason we had to make this configuration change was that during peak hours, downloads were reaching over 1,000 downloads per second.  The server simply couldn't service that many requests and the request queue would fill up and start returning "server unavailable" errors to the clients.  Adding a proxy allowed us to offload the download volume and keep the request queue from overflowing.  We used an NLB proxy "cluster" to avoid having the same request queue overflow problem on the proxy.

The other "big event" in the past month was a move of our server from our data center in Tukwila to our new data center in Quincy, WA.  I wish I could say that went smoothly.  Transferring over 8 terrabytes of data several hundred miles and building out new server infrastructure for a mission critical server is a daunting task.  We hit quite a few bumps along the way and my back side is still sore from the beatings (admittedly deservedly) I took over it.  Fortunately, we shouldn't have to do such a thing again soon.  One of my big learnings from the process though was that we need a better way to simulate our production environment in a non-production test environment.  We really needed to test all of the configuration changes we were making on a reasonably accurate simulation before trying it on the live environment.  The problem is that, to clone the hardware, it would cost close to $250K - mostly in the cost of the SAN.  Anyway, we've embarked on a process of creating such a test environment (even if it doesn't match the hardware exactly).  Hopefully this will smooth any further large scale deployment changes we make down the road.  Preventing almost 2,500 people from getting their work done is not a recipe for a long and healthy career :)

Users

• Recent users: 2,409 (up 451)
• Users with assigned work items: 4,293 (up 567)
• Version control users: 4,345 (up 525)

Work Items

• Work Items: 446,048 (up 33,117)
• Areas & Iterations: 10,536 (up 452)
• Work item versions: 3,713,236 (up 257,532)
• Attached files: 258,580 (up 14,454)
• Queries: 27,944 (up 2,291)

Version control

• Files/Folders: 312,965,192/75,535,960 (up 43,182,945/up 11,300,347)
• Total compressed file size: 2,607,236 MB (up 389,270 MB)
• Checkins: 484,546 (up 48,613)
• Shelvesets: 40,028 (up 8,611)
• Merge history: 756,402,342 (up 104,599,145)
• Pending changes: 39,586,207 (up 15,367,172)
• Workspaces: 11,415 (up 2,494)
• Local copies: 2,948,671,753 (was 2,214,366,807)

Builds

• Builds: 6,524 (up 369)

Commands (last 7 days)

• Work Item queries: 564,970 (up 106,437)
• Work Item updates: 29,854 (down 8,574)
• Work Item opens: 156,578 (down 56,465)
• Gets: 792,700 (up 470,385)
• Downloads: 52,063,240 (down 44,994,791)
• Checkins: 7,247 (up 515)
• Uploads: 159,837 (up 49,135)
• Shelves: 4,238 (up 1,134)

Brian

Another innovation in the evolution of Hotfixes

I've been keeping you all abreast of changes in our Hotfix plans.  More than a year ago, we introduced the notion of advertising hotfixes and enabling people to browse and download the hotfixes the find most useful.  Prior to that, the only way to get a hotfix was to call customer support with a problem and have them determine that one of our available hotfixes would solve it.  Six months or so ago, we introduced a new and much improved site (MSDN Code Gallery) for searching for and browsing hotfixes.  We also enabled notification via an RSS feed.

We have just added a new capability - localization of all Hotfix information.  Hotfix descriptions are now available in 10 languages (including English).  When you go to MSDN Code Gallery, you will now see a page with links for each of the supported languages.

I hope our international customers will find this to be a big help in keeping their VS, VSTS and TFS products up to date.

Brian

Expression and TFS

Warning - this is early information and is subject to change but the Expression team cleared me to say something about it.

We get a lot of feedback from customers who would like more seamless integration between Expression and the rest of their development team using TFS.  Last week I saw a demo of work being done that will result in TFS source control integration into Expression Web and Expression Blend.  It was very nice.  All of the basic operations were there - add, checkout, undo checkout, history, diff, checkin, etc.  Checkin even used the same dialog as Team Explorer so you get full fidelity - work item integration, checkin policies, etc.

It was exciting to see us making progress in this key area.  I'm told this is all going to be in the next version of Expression - of course, I don't have any release date to announce :).  However, keep your eyes open for pre-releases (like CTPs) and check out the TFS integration.  We'll be eager to hear your feedback.

Combine this with the recent release of TFS integration for Dynamics developers and we are really making progress on ensuring everyone can participate in team development regardless of what kind of development you do.

Brian

TFS on Hyper-V

With the announcement of the release of the new Microsoft Hyper-V virtualization technology, I've seen a surge of questions about TFS support for Hyper-V.  We have tested TFS running in a Hyper-V virtual machine.  It works well and is officially supported.

TFS Components & Support

One thing to keep in mind is that there are 2 major components of a TFS server.  The first is the TFS application tier.  It runs fine in Hyper-V, Virtual Server and VMWare.  You must remember that the TFS application tier in TFS 2005 and TFS 2008 is 32-bit only - so, if you are going to use it on a 64-bit Win2008 machine running Hyper-V, for example, you will need to run a 32-bit OS in its VM.  Our support team will work with you on support issues regardless of which virtualization technology you use.  However, we may ask you to reproduce problems outside of the virtual environment if we suspect it is causing a problem.  The exception to this is Hyper-V.  With Hyper-V, we are introducing full support on the virtual environment.  We (Microsoft) are also working on a certification program so that all virtualization technologies that meet the requirements can achieve the same level of support.

The second major component of a TFS server is the data tier.  The data tier is a SQLServer and the only TFS code that runs on it is TSQL stored procedures.  Of course, in a single server TFS install the application tier and the data tier run on the same machine.  My understanding is that SQL server 2005 and before do not officially support any virtualization technology.  SQL is introducing virtualization support with SQL 2008.  If you are going to use TFS with SQLServer 2008, make sure you use TFS 2008 SP1 (to be released this summer).  In general, my advice to TFS customers has been that it's fine to run the TFS application tier in a 32-bit VM but always run the SQLServer on native hardware (preferably on 64-bit if you need the system to scale).  One reason for this is for performance (see more below).  Another reason is that I have seen one or two instances of data corruption problems with SQLServer running in an improperly configured VM - the one that comes to mind was a customer who was running SQLServer in a VMWare VM and had delayed I/O enabled in the VM.  Delayed I/O is not something you want at all on a SQLServer machine - it violates key guarantees that SQL relies on to guarantee data durability.

Virtual Machine Performance Guidance

I often get asked about recommendations about hardware configuration for virtualization.  I generally focus on 2 things: memory and disk I/O.  The memory recommendation is pretty easy.  If you want to run TFS in a VM, remember that there is also a host OS running.  Sum up the memory requirements of every app in every VM you are running on the system and add about 1GB for the host OS.  I've generally found that algorithm to work well for people and I expect it will work well for Hyper-V as well.

The more complicated one is I/O.  One of the historical problems with virtualization technology is that it adds overhead when virtualizing I/O (in this case I'm mostly concerned about disk I/O).  A high scale TFS server performs a ton of I/O and any overhead can be a problem.  Fortunately, disk I/O on the application tier is generally not too high (except in the most high scale systems).  The only major source of I/O is a file cache that reduces the load on the SQLServer for commonly requested files.  TFS's use of SQLServer, however, can be very I/O intensive.  My advice to anyone running any I/O intensive application in a VM is to make sure you have a high performance disk subsystem.  Error on the side of extra disk performance.  For example, use 10K RPM or better drives, consider using RAID 10 that enables more concurrent I/O.  Make sure you have a good disk controller card.  If your system is particularly high scale, consider using a high performance SAN.

Some of this guidance may change over time with Hyper-V.  Hyper-V is a very efficient virtualization technology but I don't have enough experience with it yet to say for sure how it will affect these recommendations.  As I learn more I'll keep you up to date.

Good luck and let me know if you have good or bad experiences with TFS and Hyper-V.

Brian

Automating the creation of Team Projects

Since I announced the ability to automate the creation of Team Projects in my TFS SP1 feature set post and my March Power Tools post, I've gotten several questions about the format of the XML file you need to create and pass to:

tfpt createteamproject /settingsfile:"filename.xml"

I've attached the XML schema to this blog post and included a sample settings file.

<?xml version="1.0" encoding="utf-8"?>
<Project xmlns="ProjectCreationSettingsFileSchema.xsd">

    <TFSName>A Valid TFS server</TFSName>

    <LogFolder>
      c:\RemoteTFSProjectStarte
    </LogFolder>

    <ProjectName>
      Batch Project Creation Demo 23
    </ProjectName>
    <ProjectSiteTitle>
      <!-- default to project name if not specified -->
      Batch Project Creation Demo 23   
    </ProjectSiteTitle>
    <ProjectSiteDescription>
      <!-- default to empty string if not specified -->
      Project Description
    </ProjectSiteDescription>
    <SccCreateType>
      <!-- none | new | branch -->
      Branch
    </SccCreateType>
    <SccBranchFromPath>
      <!-- string.Empty for none and new; supplied for branch -->
      $/test44
    </SccBranchFromPath>
    <ProcessTemplateName>
      MSF for Agile Software Development - v4.2
    </ProcessTemplateName>
</Project>

Hopefully this helps make it more clear,

Brian

Rally connects to TFS for Agile project management

Today Rally is announcing that they have built a connector to enable its web based Agile project management tool to connect to TFS.  This new connection enable source code checkins and TFS builds to be tied to the project management features in Rally, enabling a more comprehensive picture of the state of your projects.

Rally is hosting a web cast on June 25th at 10AM PDT.  Click here to read more and to register for this web cast.

Here's Rally's press release: http://www.rallydev.com/press_release_080623.jsp

Brian

TFS support for Dynamics developers

Less than a month ago, Dynamics AX 2009 was released.  This new version enables Dynamics developers to store their source code and have an integrated experience for checkout, check in, get and history - the basic version control operations in TFS.  I'm excited about this because I get the question fairly often and people are always surprised when I tell them we don't have a solution.  Now I don't have to disappoint any longer.

It's great to be able to bring a whole new community of developers into the Team System collaborative development fold.  You can get a peek at what is there today by viewing this screencast.  I expect future versions of Dynamics AX will integrate additional TFS features.

If you are interested in giving this new combo a try, you can look at this whitepaper for how to set it up: http://www.microsoft.com/downloads/details.aspx?FamilyId=EFC24EDC-522E-40AA-8F36-6367ED7AB92D&displaylang=en

Let me know what you think,

Brian

PDC 2008 & precons

Preparation is in full swing for the Microsoft PDC this fall (Oct 27 - 30).  PDCs are the primary event where we talk about the future (as opposed to events like TechEd where we primarily talk about the present).  This PDC promised to have a great deal of new content for developers.  Among them, I think this will be the first conference where we really talk in depth about what is coming in the Team System release code named "Rosario".  If you are passionate about ALM, it will be a great show.

In addition to the core PDC event, there will be a set of pre-conference presentations the preceding day.  Among them will be sessions on VSTS today to position you to really understand the "tomorrow" stuff coming in the PDC.  There is also a great session on Agile development.  Check them out if you are interested.

I'm planning on being at the PDC and look forward to seeing you there.

Brian

Coolest Team System Gadget contest

Mike Azocar (one of the Team System MVPs) has come up with the idea of running a contest to see who can come up with the coolest Team System add-on.  It seems like it could be fun.  If you've got some great idea for an add-on that you've been eager to try, now might be a great time to do it and submit and see if you can win.

http://www.michaelazocar.com/blog/?p=485 

I'm trying to think of things I can provide to Mike that would make great prizes (other than lot's of money, which I can't do :)).  If you have any ideas, I'd be interested to hear them.

Brian

TFS & Teamprise for enterprises doing serious cross platform development

We recently published a case study with Thomson Reuters on their experiences adoption TFS in a cross platform development team using Teamprise for their Java developers.  It's a great testament to how the combination of these two products can work together to create a comprehensive ALM solution for a significant development team.  I frequently get asked about how well TFS + Teamprise can meet the needs of a Java development team.  I'm really glad to actually have something concrete to point at instead of having to ask you to take my word for it :)

Brian

Sharing Team System Process Templates

There are as many processes for software development as there are software teams.  Ever since we shipped TFS, I have hoped that a community would form for sharing process ideas and for sharing process templates.  Creating new process templates can be a significant undertaking.  It's way easier if you can start with a process template that's reasonably close to what you want and just tweak it.  Wouldn't it be nice if there was a library of process templates available and an easy way to share ones you create?

We'll a new CodePlex project has just been created for this purpose.  It's too early to tell whether or not it will thrive.  It will depend on how many people decide to contribute something and how actively it is managed.  If this is an area that interests you, then I encourage you to check it out.

http://www.codeplex.com/templex

Brian

DB Pro Database Extensibility

Also, at the TechEd Keynote, I demonstrated VSTS for Database Professionals working with DB2.  The first step in the process is a release we are working on now called the VSTS for Database Professionals 2008 GDR (I'm not going to waste your time trying to explain what a GDR is - just think of it as an update - not too much unlike a service pack).  This GDR will be the initial trial of a bunch of extensibility work we've been doing.  The first proof point is that SQL 2008 support is implemented on top of this extensibility and then DB2 and others can follow after that.  There are many other things in the GDR release that I think you will like.  Gert has written about them here: http://blogs.msdn.com/gertd/archive/2008/06/03/vsts-2008-database-edition-gdr-june-ctp.aspx

Brian

TechEd 2008 Keynote

I just got back from TechEd.  I had a great time and met some terrific people.  This is the first TechEd where the conference has been split across 2 weeks - the first for developers, the second for IT.

This TechEd has a first for me.  I got to do one of the big demos in big keynote with Bill - his last big keynote as full time chairman at Microsoft.  One of the things I learned in this exercise is the amount of work that goes into putting this together.  The work on the demo, the rehearsals, getting everyone to agree on the talking points, all of the back stage logistics.  I'd estimate I spent days preparing for my little 7 minute appearance.  It was fun though :)

In my mind (admittedly biased), the demo was terrific.  That's not to say my delivery was particularly amazing but rather the demo itself was very good and the technologies we were showing are just amazing.  The demo I did basically highlighted two things - the new direction we are headed in architecture tools and modeling and the work we are doing on DBPro extensibility along with the extension IBM is building that enables Visual Studio Team System for Database Professionals to give you the same great experience against a DB2 database that you can get against a SQL server database.

The architecture tools we are building are so unbelievably exciting to me that I can't stand it.  I'm wishing I'd have had these tools years ago.  I can tell you for certain that we could have shaved months off of the development of TFS over the past few years if we had had them.

The thing that has me the most excited is a new feature called a layer diagram.  This layer diagram allows you to design the logical layering of your application and specify the dependencies you intend to have.  For example, you wouldn't want your client to have a dependency on you data layer in a 3 tier app.  You wouldn't want your server business logic taking a dependency on something in the Windows.Forms assembly, etc.

Once you have this logical layering, you can associate the layers with specific assemblies, namespaces or classes in your actual application.  Now, the magic happens.  VSTS can parse/reverse engineer your code to determine the "actual" dependencies in your application.  Having done this, it can compare the actual to the "desired" as specified in the layer diagram and let you know about any problems you have.  Going even further, you can configure a checkin policy to validate this on every single checkin.  This way you can prevent people from checking in spaghetti architecture in the first place.

It's very hard to communicate an architecture to everyone on the team.  It's even harder to prevent them from accidentally violating the architecture.  These tools are a huge leap forward in ensuring that you end up with the architecture you designed.

On top of this cool new layer diagram and validation is another feature called the "Architecture Explorer".  It is a very handy tool for visualizing the architecture your code implements today.  It allows you to graphically browse through your application and its dependencies and understand them.  As you update your code, you can see the Architecture Explorer diagrams change.

All of this (and more) represent a pretty big shift in our approach to the VSTS Architecture SKU towards tools that are applicable to a broader set of problem domains and compelling to a larger constituency of users.  It catapults the architecture SKU from a moderately interesting overall value to one the the most exciting things in Team System.  I can't wait for Rosario to become "go-live dogfoodable" so that we can use this new stuff in our daily jobs.

Brian

Translating Team System Web Access to your langauge

We have gotten some requests for translations of Team System Web Access into additional languages (beyond the 9 we already do).  I am speculating that the additional interest here comes from the fact that TSWA targets a broader audience beyond the core development team and as such translations are even more valuable than for Visual Studio itself.

We aren't set up to create and test these translations, however, we would like to satisfy the demand.  Fortunately, translating TSWA is not a massive undertaking.  In all, it only involves translating a few hundred strings.  For our own localization effort, we have already separated these strings into separate .resx files that are compiled into satellite assemblies.  In short this means that if we give out the .resx files and a relatively short instructional document, it should be reasonably straight forward for you to localize TSWA into any language you choose.

I'd also like to make it possible for you to contribute your translations back to the community for others to use if you choose to do so.

Our thinking is that we will create a CodePlex project for TSWA translations.  We will upload all of the .resx files for the 9 languages we currently support and a document explaining how to create and install translations.  You can download the files, translate, build, install, test, and use them.  If you choose to contribute your translations back to the community, we will provide a way for you to request contributor rights to the CodePlex project and you will be able to upload your translation to CodePlex for others to download.

We're still working through the plans but this is the rough idea.  I expect it will take a few weeks to get this all set up - we're still working out the schedule so I'm not sure but I don't think it's going to take a long time.

I hope this will be helpful to you.  If you have any thoughts and/or comments, I'd love to hear them.  Keep an eye on my blog for further announcements as we get this ready.

Brian