Allen Clark

November Feedback on Help for Team Foundation 2010 Beta 2

Allen Clark — Wed, 09 Dec 2009 00:16:00 GMT

Summary Page views continue to increase Ratings continue to be high (average 4.31 out of 5, compared to about 3 traditionally) Project management continues to have twice as many page views as the next area (also still administration) 4 out 5 of the most...(read more)

October Feedback on Team Foundation Help in Dev 10 Beta 2

Allen Clark — Fri, 06 Nov 2009 23:42:00 GMT

Last month, Beta 2 shipped, and we posted a corresponding update to the help content. Page views, ratings, and feedback have all more than doubled in October compared to previous months. Overall ratings remained higher than we had in VS2005 & VS2008, and most of the lower ratings come from topics that are effectively unchanged from those older versions. That's a good indicator that the changes we've made to be user-centric and minimalist are working.

Looking inside, we see that growth is most dramatic in the project management content. In VS2005 & VS2008, the most viewed content were in administration, and project management was the second most viewed with about 1/3 as many page views. When we shipped Beta 1 content, project management became the most viewed area of content by a small margin. In October, though, project management content was viewed twice as often as the next most viewed area - administration. (So far in November, it's actually running at 3x the page views of administration). This suggests that bringing process guidance and help together is making our content more relevant, as we had hoped.

It's also interesting that, although the agile guidance topics make up a fairly small portion of the content, 19 out of 20 of the most viewed topics in project management were agile guidance topics.

Ratings and Feedback

MSF for Agile Software Development v5.0 recieved the highest rating (5) and this comment: "Great stuff".

Customizing and Managing Work Item Types recieved the highest rating (5) and a comment in another language that I was unable to translate.

Iteractions between SharePoint Products and Team Foundation Server recieved the highest rating (5) twice.

Creating a Team Project, Product Backlog Workbook, and User Story all recieved high (4) ratings. User Story recieved this comment about the help experience:

In LigthWeight-view there could be a thin vertical bar left to the navigation history frame entries with an arrow to the bottom. This way it would me more intuitive that the upper left frame contains the navigation history...

Configure Settings for Dashboard Compatibility recieved an average (3) rating and this comment:

Should make it clear that Excel Services only work on MOSS 2007 Enterprise. Single sign-on doesn't seem to work if MOSS is installed on a workgroup rather than a domain. I also haven't got Shared Services Administration working, but I don't know whenther this is workgroup-related or not.

Team Foundation Server Default Groups, Permissions, and Roles, and Team Foundation Server Permissions recieved low (2) ratings.

Share a Work Item Query recieved the lowest rating (1), and this comment: "How do I open it once I receive the query in email???". This is a topic that was touched by some reorganization, but is very similar to pre-Dev 10 content. It seems that we could have caught this if we would have analyzed the customer scenario for this content more effectively.

Enabling New Application Lifecycle Management Features for Visual Studio 2010 Beta 2 in Upgraded Team Projects

Allen Clark — Tue, 13 Oct 2009 16:00:00 GMT

After you upgrade your server to Visual Studio Team System 2010 Beta 2, you have a wealth of new features available. Generally, you don’t have to do anything except start using those new features. For example, you can use the new types of links between work items, such as parent and child links, immediately after the server is upgraded.

In the cases described here, though, you’ll have to do some work if you want to use the new features in team projects that were upgraded from an earlier version of Visual Studio Team System. You can enable these features immediately after upgrading your server, or you can begin using your team project and enable any or all of them at a more convenient time in the future.

The following posts provide manual steps for enabling each of these features. Hakan Eskici has also provided a script that you may find helpful, especially if you are using MSF for Agile Software Development v4.2 without customizations. Even if you've customized your process, you should be able to tweak the script and use it.

Enable Test Management: Visual Studio Team System 2010 includes features that allow you to manage your tests more effectively. The screenshots below show the new test case, and the integration of the test steps with a bug. Read Chris Patterson’s blog post to learn how to add test management to team projects that you upgraded to Visual Studio Team System 2010.

Enable Lab Management: You can set up environments to run tests and collect data remotely. For information about enabling lab management, read Vishal Mehrotra’s blog post.

Enable the Agile Planning Workbooks: You can use the new planning workbooks that are included in MSF for Agile Software Development v5.0 to help you plan your project and iterations. If your team project uses MSF for Agile Software Development v4.2, read Aaron Bjork’s blog posts to enable the iteration backlog and product backlog new workbooks.

Enable Reports: If you open your team project's Reports folder, you'll find that the reports you had been using aren't there. They are still available at http://<server>/reports/<team project>, but they will need to be updated to adapt to the changes to the schema of the cube and the warehouse database. These reports will continue to show the data as it existed before your server was upgraded. Aaron Bjork has a post that explains where the reports can be found and how to move them to the appropriate location on the report server. John Socha-Leialoha has posted information to help you make the changes to your reports so that they work with the new schema and pick up the new data from the upgraded server. Sunder Raman has posted a good article on what those changes are.

September Feedback on Help Content for Beta 1 of Team Foundation

Allen Clark — Sat, 10 Oct 2009 01:45:00 GMT

In September, we didn't have a lot of feedback, so I'm going with the flow and providing a very brief overview of that feedback. (Excuses, excuses). In fact, only three topics recieved ratings.

Visual Studio Team System received two ratings with an average score of 3 (average). I'm not sure whether they were both actually 3's.
Work Item Types received a rating of 1 (lowest), and the comment "clearly not up to date". That's true. The example we provide still uses the version of the MSF for Agile work item types from VS2008.
Using the Checkin and Pending Changes Window recieved a rating of 5 (highest). It's the first time I've seen a 5 on one of the topics that we carried over from VS2008. Interestingly, it looks more like our newer content than most of the other content from 2008. Even though the title is based on a feature, the internal content is focused on what you can do, and it provides conceptual information to allow them to understand whether each of the linked documents is what you are looking for. It's like a conceptual navigation topic without the table.

Project management content was viewed substantially more than any other section this time. It was hit 6710 times compared to 4540 for administration, the second most frequently viewed section.

August Feedback on Beta 1 Content for Team Foundation 2010

Allen Clark — Wed, 02 Sep 2009 07:28:00 GMT

Summary

Admin content is once again the most viewed section, although not by the historical 5:1 margin.
The trend of high ratings for new content (using new approaches) continues.

Details

In August, there were almost 15,000 page views, with 9 ratings (mostly high) and 3 comments. That's about the same as last month.

Team System

Our three high volume topics in the Team System project are still getting substantial traffic. The top level Visual Studio Team System topic received two highest (5) ratings, and the What's New topic recieved two average (3) ratings. That's very inconsistent with what we've seen in the past - high ratings for What's New and low ratings for the top level VIsual Studio Team System topic.

Team Foundation

Admin topics are again the most read section. Project management topics are still a much higher proportion of the total page reads than the traditional 1/5th. Ratings for admin & project management were good.

Admin

Two command line reference topics recieved the highest (5) ratings and comments. However, the comments were spam (a web site and a string of repeated x's), and the page views are low, so I won't read anything into those.

Project Management

Project management received two highest (5) rankings and one average (3) ranking with feedback.

Planning and Tracking Projects and Query Fields, Operators, Values and Variables recieved the highest (5) rating. Planning and Tracking Projects contains much of the content that has become the Scrum Processes section of the agile process guidance in Beta 2. The topic on query fields, etc is an exhaustive reference topic rather than a scenario-based topic. The latter suggests that there are times when detailed reference info is still valued.

The walkthrough topic recieved an average (3) rating, which is low for topics that are new or rewritten for Dev 10. The feedback indicated (correctly) that the topic isn't consistent with agile practices. That sparked a conversation resulting in a new goal for work item tracking content: we will align with agile practices like iterative development, but not necessarily with the MSF for Agile process template. We're not going to do a wholesale change to realign work item tracking content before RTM, but we may have a time period after RTM where we target that alignment. In this case, the reviewer left his email, and we've been in contact with him.

Watching from Previous Months

There are a few trends that we're watching from previous months.

Page views: Historically, admin has had about 5 times as many page views as project management. In the last few months, project management has had more than admin. This month, project management had about 20% fewer than admin.
Empty topics: We shipped some empty topics and recieved early feedback that customers didn't like that. However, that hasn't been repeated since that first month.
Localization: There's no new data on localization.

Feedback on Help for Team Foundation

Allen Clark — Fri, 21 Aug 2009 08:25:00 GMT

Include your email when you leave feedback on help for project management in VSTS if you'd like a response. I can't guarantee that you'll get one, but I'd wager that you will.

Great Organizations Surprise You

Allen Clark — Wed, 12 Aug 2009 02:17:00 GMT

As we write help for Team System, we try to hand topics off for localization as early as we can, but not too early. We don't want to incur the cost of relocalizing topics that we need to change. For example, if I hand off a topic that I've written about the warehouse schema, and then the schema changes, I'm going to have to update that. Now, localization takes advantage of localization memory, so it shouldn't be a big deal; just the changes get relocalized, right? Not so, at least until recently. Localization memory wasn't built from these incremental handoffs. Instead, it was built only at major releases. That meant that we had to do a lot of educated guessing about what features might change between the first handoff and RTM.

A few days ago, I heard that we would have an early loc hand-off, and I started thinking about what was complete enough to hand off and, of that, which of the features are stable enough to avoid the risk of change. But it turns out our loc team is now getting the loc memory updated with each drop, so we can hand off topics that without really worrying a lot about relocalizing them. That takes a whole lot of guesswork out of my planning - I can just write till it's done, and hand it off.

That's the kind of thing I've come to expect working on Team System - that someone has done something smart and now it's a little easier for me to focus on the writing and less on the overhead of getting great content out there. Nice.

July Feedback on Beta 1 Content for Team Foundation 2010

Allen Clark — Mon, 03 Aug 2009 20:26:00 GMT

Summary

The trends this month are good (or neutral).

New content is getting high ratings; existing content is taking the low ratings.
No new negative feedback on empty topics; one high rating on an "empty" navigation topic.
Project management remains 5 times higher than it's traditional share of page hits, on par with administration

Details

We recieved 7 ratings and 1 comment in July, mostly good, which is encouraging after last month. Still, we need to see how this holds up.

Team System

In Team System, the What's New topic was viewed almost 1000 times and it recieved another highest (5) rating.

Team Foundation

This month, the feedback was all build and version control. Project management (the top page views and the only ratings last month), recieved no ratings, and remained the most read section, but just barely.

TFSAdmin: How to: Set Up and Manage a Build System recieved the highest (5) rating.
dv_TeamBuild: both Define Workflow Builds using the Default Template, and How to: Create a Basic Build Definition recieved high (4) ratings.
dv_TeamVersionControl: How to: View Where and When Changesets have been Merged recieved the highest (5) rating, and How to: Cloak and Uncloak Folders in a Workspace recieved two low (2) ratings.
dv_TeamProject: How to: Create an Office Project Plan from Team Foundation Work Items recieved this comment: "Good one".

All of the topics that recieved high ratings are either new or significantly reworked for Dev 10. The topic that recieved low ratings is unchanged from previous releases, except for maintenance of links.

Define Workflow Builds using the Default Template really only introduces two other topics, and makes the general statement that it will be fleshed out with more info later. I'm guessing that the reader was satisfied with the topics under this one, and not put off that it only contained those two links becaue it is Beta content, and we explained that the topic will be fleshed out later.

Watching from Last Month

We saw some potential trends last month. Here's how this month's data relates to that.

Page Views: Project management, which traditionally gets 1/5 the page views of administration (traditionally the most viewed section). Last month, proportion of page views for most projects remained pretty consistent wiht the traditional distribution, except that dv_TeamProject moved to the top of the stack, with almost 50% more page views than even TFSAdmin. This month, project management is almost exactly the same as TFSadmin, still substantially above it's traditional share.
Empty Topics: The was no new negative feedback from the empty topics, and there was one high rating for a links-only topic with the disclaimer. It's still clearly too early to know how this practice is being accepted.
Loc: There's no new data on localization

June feedback on Beta 1 content for Team Foundation 2010

Allen Clark — Fri, 24 Jul 2009 23:00:00 GMT

Earlier this month, I sent this email internally summarizing the feedback we've recieved on the Beta 1 content for Team Foundation.

Give us feedback

Allen Clark — Thu, 18 Jun 2009 01:19:00 GMT

We published help content for Visual Studuio Team System 2010 Beta 1 that is substantially different from what we published in VSTS 2008. We're anxious to hear what worked well for you and what didn't. If you rate the topics that you read, we will use that to understand how to continue to improve the help content. If you have specific comments, please add those. Comments about topics are especially useful, but don't feel like you have to add a comment to rate the topic. Most people, by a vast majority, don't comment or even rate the topics they read. If you do take the time to leave us feedback, you're like to be among only a handful that have done so for that topic, so your feedback will absolutely not be lost in the crowd. I look at the feedback regularly and use that to determine my backlog of work, and that's pretty typical for writers working on VSTS.

Customizing work items

Allen Clark — Tue, 19 May 2009 00:02:00 GMT

Do you customize the work items that you use with Team Foundation? Kathryn Elliot is looking for your input on what sort of documentation would be most helpful on her blog.

Enabling New Features of Visual Studio Team System 2010 Beta 1 in Upgraded Team Projects

Allen Clark — Fri, 15 May 2009 21:23:00 GMT

After you upgrade your server to Visual Studio Team System 2010 Beta 1, you have a wealth of new features available. Generally, you don’t have to do anything except start using those new features. For example, you can use the new types of links between work items, such as parent and child links, immediately after the server is upgraded.

The following posts provide manual steps for enabling each of these features. Hankan Eskici has also provided a script that you may find helpful, especially if you are using MSF for Agile Software Development v4.2 wihtout customizations. Even if you've customized your process, you should be able to tweak the script and use it.

Enable Test Management: Visual Studio Team System 2010 includes features that allow you to manage your tests more effectively. The screenshots below show the new test case, and the integration of the test steps with a bug. Read Chris Patterson’s blog post to learn how to add test management to team projects that you upgraded to Visual Studio Team System 2010.

Enable Lab Management: You can set up environments to run tests and collect data remotely. For information about enabling lab management, read Vishal Mehrotra’s blog post.

Enable the Agile Planning Workbooks: You can use the new planning workbooks that are included in MSF for Agile Software Development v5.0 to help you plan your project and iterations. If your team project uses MSF for Agile Software Development v4.2, read Aaron Bjork’s blog posts to enable the iteration backlog and product backlog new workbooks.

Enable Branch Visualization: Visual Studio Team System 2010 makes a clear distinction between folders and branches, and adds some new tools to visualize your branch hierarchy and initiate actions such as merges. Read Jamie Cool’s blog post to learn how to convert folders into branches.

Enable Reports: If you open your team project's Reports folder, you'll find that the reports you had been using aren't there. They are still available at http://<server>/reports/<team project>, but they will need to be updated to adapt to the changes to the schema of the cube and the warehouse database. These reports will continue to show the data as it existed before your server was upgraded. Aaron Bjork has a post that explains where the reports can be found and how to move them to the appropriate location on the report server. John Socha-Leialoha has posted information to help you make the changes to your reports so that they work with the new schema and pick up the new data from the upgraded server. Sunder Raman has posted a good article on what those changes are.

Reporting Progress

Allen Clark — Fri, 15 May 2009 02:51:00 GMT

Since we mirror our topics from our content management system into TFS, we're able to use the TFS warehouse and Excel to easily create reports like this chart.

We also link the topics to the the deliverables that describe the work that the product unit is doing. That lets us create charts that show progress by the area of the product, like this.

Each of these took just a few minutes to create using Excel and the techniques described in How to: Create a Report in Microsoft Excel. Now, we just refresh the report in Excel (Data menu, Refresh All) to see the current status.

Managing Technical Reviews in TFS and SharePoint

Allen Clark — Wed, 01 Apr 2009 23:59:00 GMT

I've alluded to my plans to describe how we use TFS to plan, track progress, and do things like technical reviews. Brian Harry mentioned our tech review process in his blog post, so I'll write about that first.

In the past, we've used an internal tool to manage technical reviews. The tool has some nice features - it renders content pretty similar to the way our help system does, and you can post comments directly in the topic. For a long time, though, there's been a debate over whether the good things outweigh the bad. Recently, support of the internal tool withered away, so we created this new process. As I've mentioned before, each of our help topics is represented in TFS as a work item for our own planning purposes. That gave us a good starting point to build a new tech review process. We have very little investment in the tools to support this process because it uses TFS and SharePoint instead of a specialized internal tool.

SharePoint

We post word documents to a document library and the reviewers check them out and add their comments and suggested changes. This still has some flaws. It means only one reviewer can comment at a time, and we have some challenges getting the content from our content management system into a word document with the art and all the styles reasonably rendered. Still, the product team really prefers providing feedback this way so, even with some strange formatting and the occasional missing art, this has been successful for us.

TFS

Since we already have work items in TFS that represent each of our help topics (and since these are kept up to date automatically), we just tagged on a few more fields and created some supporting queries to help everyone find the topics that they need to review. Here's the tech review tab. With the exception of State, these are the fields we added to support tech reviews. (Review is the URL to the document on the SharePoint site).

We created a query that lets anyone on the team find the topics that they need to review.

AndOr Field Operator Value

Team Project = @Project

And Work Item Type = Help Topic

And State = Active

And DS Topic Status = In Tech Review

And Primary Reviewer = @Me

And Reviewed by Primary <> Reviewed

Or Additional Reviewers = @Me

And Reviewed by Additional <> Reviewed

Or Additional Reviewer2 = @Me

And Reviewed by Additional2 <> Reviewed

Or Additional Reviewer3 = @Me

And Reviewed by Additional3 <> Reviewed

Reports

We were able to use the TFS warehouse to create some helpful reports. For example, we created a similar query that returns all topics in review, sorted by due date. We used that query to create a work item list in Excel, applied some conditional formatting to highlight topics nearing the due date or late, and used that in regular nag mail.

How we track content work.

Allen Clark — Tue, 10 Mar 2009 22:52:00 GMT

For a while, we were tracking our work by creating a backlog of scnearios, prioritizing them, then planning the top scenarios for our next iteratoin of work. We broke the scenarios down into tasks and tracked progress on those tasks over the course of an iteration. I mentioned that in my last post. We've refined that. We still use scenarios to design our content, but we break those down into new or changed topics, and then we track the progress of those topics. It's great because we can measure our progress in terms of our deliverables. That's harder for software projects because the deliverable unit isn't always so distinct, but for us, tracking progress on topics is just terrific.

There was one challenge - how do we report on the progress using topics as the unit that we track? We have a content management system that provides limited access for reporting and, because it's used by many teams across Microsoft, is difficult to enhance to meet a single team's needs. We decided to mirror our content into TFS and add the metadata that we need for our tracking purposes. That's turned out to be a great decision. Because our topics are mirrored to TFS, we can asscoiate them with the planning artifacts that the product team uses. We can now explain our plans, demonstrate our progress, and express our needs (like technical reviews) more effectively. I'll write a post on each of those points soon.