Allen Clark

More about Pageviews for Project Management and Visual Studio Application Lifecycle Management

The last time I posted, I pointed out the significant increase in pageviews for project management as a share of pageviews for all the content we have for Visual Studio ALM (fka VSTS). I know that pageviews for TF administration content are down, and that's at least in part due to the new administration console for Team Foundation, and they way it services F1 requests. First, if you're using the admin console for TF in Dev 10, then you are probably using a UI to do something that you had to do with command line tools in VS 2005 & VS 2008. If you needed help beyond what the command line tool gave you with /?, you probably searched online and, hopefully, found the topics you needed on MSDN. Now, when you're in the administration console, you have the option of hitting F1 instead of searching manually online for the information you need. If you do that, you'll be directed to a local chm file instead of the online help files (even if you have online help configured). I think that's probably the primary reason the page counts for administration content has dropped off.

So, I wondered whether that drop in counted pageviews for the admin topics was contributing significantly to the apparent increase in project management topics. I decided to look at the pageviews for project management as a share of VS ALM both with TF administration and without it to see. I also broke out the other areas of VS ALM (like testing) for comparison. Here's the first chart, which includes TF administration. The big changes for Dev 10 are a drop in TF administration, and significant increases for both project management and architecture. (I hadn't noticed the increase for architecture before).

 

So, how does it look if administration page views aren't included? Pretty much the same for project management. Project management's increase in share pageviews isn't really impacted by taking TF administration out of the mix.

I noticed a few other things in the chart that are interesting, or at least worth explaining.

• As I mentioned above, architecture has seen a substantial increase in the share of pageviews.
• Testing, project management, and architecture each have about a 20% share of the pageviews.
• Team data dropped off to nearly nothing, but that's because it's mostly folded into developer topics.
•  Combined, data & developer took a pretty sharp drop in share of pageviews, as did version control.
• Testing pageview share also dropped, but less than data and version control.
• Lab appears in Dev 10 for the first time because there was no lab product in VS 2005 & VS 2008.

Project Management page views

I've commented several times that page views for project management content are way up in Dev 10. I decided to look at that a little more closely. I looked at the page views for project management as a percentage of all of VSTS (or VS ALM). For comparison, I also looked at the page views for the rest of Team Foundation. The chart below shows that project management topics, which have traditionally received 3% of the page views in this set of content, is now getting 18% of the page views, and is now receiving as many page views as all other content for Team Foundation.

I've wondered whether the newness of the content contributes to that. It may, to some extent. However, all areas of VS ALM have significant sections of new content, so it's hard to attribute the change to newness. When I look at where the high page views are occurring, they are mainly in the handful of topics in the agile process guidance: MSF for Agile Software Development v5.0. This is content that we didn't have in the help system for VS 2005 & VS 2008. Instead, we had very generic topics about things like assigning work items. When we decided that, in Dev 10, we would focus on what the user does rather than what the product does, we started to see that we needed to write about user stories and tasks and product backlogs instead of "work items". In VS 2005 & VS 2008, though, these subjects were handled outside the help system in a process guidance implementation that was completely disconnected from help. In Dev 10, we brought the agile process guidance into the help system and completely reworked it with the help of several agile experts. Although these topics are only a small portion of all of our topics on project management, they are the most viewed topics and contribute more than any other section to the growth of project management content page views. I believe that these page view numbers are strong evidence that this change is doing what we had hoped - making the project management content more relevant to those of you using VS ALM to manage your projects.

Opposing Trends in Feedback on my Help Topics for Team Foundaiton

When I look at the feedback that we've received for the topics that I wrote or that I've picked up to maintain, I see a generally positive trend. In VS 2008, ratings for these topics are slightly better than they are for my topics in VS 2005. In Dev 10 Beta, the ratings are vastly improved over that. However, when I separate the SDK topics, I see that there are two distinct trends.

The trend for the mainstream topics (Team Foundation in the chart) is great. We made a little headway in VS 2008, and it looks like we're firing on all cylinders for Dev 10. That's encouraging, especially since that's where I've invested almost all of my time and innovation. However, the trend for the SDK topics is really bad. We haven't published SDK topics for Dev 10 betas, so there's no feedback. However, the trend from VS 2005 & VS 2008 indicates that, unless we change course, we should not expect good results. So, we've ramped up our efforts and we have a substantial effort under way that involves both the writing team and the API developers. When we publish the TFS API reference for Dev 10, I hope it turns the trend in the right direction. We'll continue to watch the feedback, learn what is working, and work to make the API reference excellent as well.

November Feedback on Help for Team Foundation 2010 Beta 2

Summary

• Page views continue to increase
• Ratings continue to be high (average 4.31 out of 5, compared to about 3 traditionally) 
• Project management continues to have twice as many page views as the next area (also still administration)
• 4 out 5 of the most viewed topics were in project management
• We have positive feedback regarding our approach of delivering placeholder topics in the beta releases.

Details 

In November, we had another substantial increase in page views, and we're still getting mostly positive feedback (averaging 4.31 out 5).

Four of the five most commonly viewed topics were from project management, which is still getting about twice as many views as the next most viewed section, administration.

 

Project Management 

Six of the topics in project management received ratings, all high (4) or highest (5).

 

Administration

Two administration topics received high (4) ratings and comments. 

Add a Report Server to a Team Project Collection is a placeholder topic that simply indicates the intent to publish this by RTM. In this case, we asked for feedback about whether that is helpful. The verbatim feedback is:

Having a placeholder is great in conveying that 'we haven't forgotten this topic' it's just not ready. Would be great if there was some inkling of what info you would provide once its done. That way if it doesn't seem inline with what I was looking for I could provide that feedback before hand as well and help make the topic more relevant hopefully!

Lab Management System Requirements received a high (4) rating, and the comment: "usefully".

 

Version Control

Version control content received a high (4) and average (3) rating.

Use Labels to Take a Snapshot of Your Files received an average (3) rating and this comment:

Are you sure the rights to modify a label is Administer Shelves? In the admin security page the following right still existsAdminister labels tf: LabelOther Users who have this permission can edit or delete labels created by another user.

Build

There was no feedback for build content.

 

October Feedback on Team Foundation Help in Dev 10 Beta 2

Last month, Beta 2 shipped, and we posted a corresponding update to the help content. Page views, ratings, and feedback have all more than doubled in October compared to previous months. Overall ratings remained higher than we had in VS2005 & VS2008, and most of the lower ratings come from topics that are effectively unchanged from those older versions. That's a good indicator that the changes we've made to be user-centric and minimalist are working.

Looking inside, we see that growth is most dramatic in the project management content. In VS2005 & VS2008, the most viewed content were in administration, and project management was the second most viewed with about 1/3 as many page views. When we shipped Beta 1 content, project management became the most viewed area of content by a small margin. In October, though, project management content was viewed twice as often as the next most viewed area - administration. (So far in November, it's actually running at 3x the page views of administration). This suggests that bringing process guidance and help together is making our content more relevant, as we had hoped.

It's also interesting that, although the agile guidance topics make up a fairly small portion of the content, 19 out of 20 of the most viewed topics in project management were agile guidance topics.

Ratings and Feedback

MSF for Agile Software Development v5.0 recieved the highest rating (5) and this comment: "Great stuff".

Customizing and Managing Work Item Types recieved the highest rating (5) and a comment in another language that I was unable to translate.

Iteractions between SharePoint Products and Team Foundation Server recieved the highest rating (5) twice.

Creating a Team Project, Product Backlog Workbook, and User Story all recieved high (4) ratings. User Story recieved this comment about the help experience:

In LigthWeight-view there could be a thin vertical bar left to the navigation history frame entries with an arrow to the bottom. This way it would me more intuitive that the upper left frame contains the navigation history...

Configure Settings for Dashboard Compatibility recieved an average (3) rating and this comment:

Should make it clear that Excel Services only work on MOSS 2007 Enterprise. Single sign-on doesn't seem to work if MOSS is installed on a workgroup rather than a domain. I also haven't got Shared Services Administration working, but I don't know whenther this is workgroup-related or not.

Team Foundation Server Default Groups, Permissions, and Roles, and Team Foundation Server Permissions recieved low (2) ratings. 

Share a Work Item Query recieved the lowest rating (1), and this comment: "How do I open it once I receive the query in email???". This is a topic that was touched by some reorganization, but is very similar to pre-Dev 10 content. It seems that we could have caught this if we would have analyzed the customer scenario for this content more effectively.

 

Enabling New Application Lifecycle Management Features for Visual Studio 2010 Beta 2 in Upgraded Team Projects

After you upgrade your server to Visual Studio Team System 2010 Beta 2, you have a wealth of new features available. Generally, you don’t have to do anything except start using those new features. For example, you can use the new types of links between work items, such as parent and child links, immediately after the server is upgraded.

In the cases described here, though, you’ll have to do some work if you want to use the new features in team projects that were upgraded from an earlier version of Visual Studio Team System.  You can enable these features immediately after upgrading your server, or you can begin using your team project and enable any or all of them at a more convenient time in the future.

The following posts provide manual steps for enabling each of these features. Hakan Eskici has also provided a script that you may find helpful, especially if you are using MSF for Agile Software Development v4.2 without customizations. Even if you've customized your process, you should be able to tweak the script and use it.

Enable Test Management: Visual Studio Team System 2010 includes features that allow you to manage your tests more effectively.  The screenshots below show the new test case, and the integration of the test steps with a bug. Read Chris Patterson’s blog post to learn how to add test management to team projects that you upgraded to Visual Studio Team System 2010.

Enable Lab Management: You can set up environments to run tests and collect data remotely. For information about enabling lab management, read Vishal Mehrotra’s blog post.

Enable the Agile Planning Workbooks: You can use the new planning workbooks that are included in MSF for Agile Software Development v5.0 to help you plan your project and iterations. If your team project uses MSF for Agile Software Development v4.2, read Aaron Bjork’s blog posts to enable the iteration backlog and product backlog new workbooks.

Enable Reports: If you open your team project's Reports folder, you'll find that the reports you had been using aren't there. They are still available at http://<server>/reports/<team project>, but they will need to be updated to adapt to the changes to the schema of the cube and the warehouse database. These reports will continue to show the data as it existed before your server was upgraded. Aaron Bjork has a post that explains where the reports can be found and how to move them to the appropriate location on the report server. John Socha-Leialoha has posted information to help you make the changes to your reports so that they work with the new schema and pick up the new data from the upgraded server. Sunder Raman has posted a good article on what those changes are.

September Feedback on Help Content for Beta 1 of Team Foundation

In September, we didn't have a lot of feedback, so I'm going with the flow and providing a very brief overview of that feedback. (Excuses, excuses). In fact, only three topics recieved ratings.

• Visual Studio Team System received two ratings with an average score of 3 (average). I'm not sure whether they were both actually 3's.
• Work Item Types received a rating of 1 (lowest), and the comment "clearly not up to date". That's true. The example we provide still uses the version of the MSF for Agile work item types from VS2008.
• Using the Checkin and Pending Changes Window recieved a rating of 5 (highest). It's the first time I've seen a 5 on one of the topics that we carried over from VS2008. Interestingly, it looks more like our newer content than most of the other content from 2008. Even though the title is based on a feature, the internal content is focused on what you can do, and it provides conceptual information to allow them to understand whether each of the linked documents is what you are looking for. It's like a conceptual navigation topic without the table.

Project management content was viewed substantially more than any other section this time. It was hit 6710 times compared to 4540 for administration, the second most frequently viewed section.

August Feedback on Beta 1 Content for Team Foundation 2010

Summary

• Admin content is once again the most viewed section, although not by the historical 5:1 margin.
• The trend of high ratings for new content (using new approaches) continues.

Details

In August, there were almost 15,000 page views, with 9 ratings (mostly high) and 3 comments. That's about the same as last month.

Team System

Our three high volume topics in the Team System project are still getting substantial traffic. The top level Visual Studio Team System topic received two highest (5) ratings, and the What's New topic recieved two average (3) ratings. That's very inconsistent with what we've seen in the past - high ratings for What's New and low ratings for the top level VIsual Studio Team System topic.

Team Foundation

Admin topics are again the most read section. Project management topics are still a much higher proportion of the total page reads than the traditional 1/5th. Ratings for admin & project management were good. 

Admin

 Two command line reference topics recieved the highest (5) ratings and comments. However, the comments were spam (a web site and a string of repeated x's), and the page views are low, so I won't read anything into those.

Project Management

 Project management received two highest (5) rankings and one average (3) ranking with feedback.

Planning and Tracking Projects and Query Fields, Operators, Values and Variables recieved the highest (5) rating. Planning and Tracking Projects contains much of the content that has become the Scrum Processes section of the agile process guidance in Beta 2. The topic on query fields, etc is an exhaustive reference topic rather than a scenario-based topic. The latter suggests that there are times when detailed reference info is still valued.

The walkthrough topic recieved an average (3) rating, which is low for topics that are new or rewritten for Dev 10. The feedback indicated (correctly) that the topic isn't consistent with agile practices. That sparked a conversation resulting in a new goal for work item tracking content: we will align with agile practices like iterative development, but not necessarily with the MSF for Agile process template. We're not going to do a wholesale change to realign work item tracking content before RTM, but we may have a time period after RTM where we target that alignment. In this case, the reviewer left his email, and we've been in contact with him.

Watching from Previous Months

There are a few trends that we're watching from previous months.

• Page views: Historically, admin has had about 5 times as many page views as project management. In the last few months, project management has had more than admin. This month, project management had about 20% fewer than admin.
• Empty topics: We shipped some empty topics and recieved early feedback that customers didn't like that. However, that hasn't been repeated since that first month.
• Localization: There's no new data on localization.

Feedback on Help for Team Foundation

Include your email when you leave feedback on help for project management in VSTS if you'd like a response. I can't guarantee that you'll get one, but I'd wager that you will.

Great Organizations Surprise You

As we write help for Team System, we try to hand topics off for localization as early as we can, but not too early. We don't want to incur the cost of relocalizing topics that we need to change. For example, if I hand off a topic that I've written about the warehouse schema, and then the schema changes, I'm going to have to update that. Now, localization takes advantage of localization memory, so it shouldn't be a big deal; just the changes get relocalized, right? Not so, at least until recently. Localization memory wasn't built from these incremental handoffs. Instead, it was built only at major releases. That meant that we had to do a lot of educated guessing about what features might change between the first handoff and RTM.

A few days ago, I heard that we would have an early loc hand-off, and I started thinking about what was complete enough to hand off and, of that, which of the features are stable enough to avoid the risk of change. But it turns out our loc team is now getting the loc memory updated with each drop, so we can hand off topics that without really worrying a lot about relocalizing them. That takes a whole lot of guesswork out of my planning - I can just write till it's done, and hand it off.

That's the kind of thing I've come to expect working on Team System - that someone has done something smart and now it's a little easier for me to focus on the writing and less on the overhead of getting great content out there. Nice.

July Feedback on Beta 1 Content for Team Foundation 2010

Summary

The trends this month are good (or neutral).

• New content is getting high ratings; existing content is taking the low ratings.
• No new negative feedback on empty topics; one high rating on an "empty" navigation topic.
• Project management remains 5 times higher than it's traditional share of page hits, on par with administration

 

Details

We recieved 7 ratings and 1 comment in July, mostly good, which is encouraging after last month. Still, we need to see how this holds up.

 

 

 

Team System

In Team System, the What's New topic was viewed almost 1000 times and it recieved another highest (5) rating.

 

 

Team Foundation

This month, the feedback was all build and version control. Project management (the top page views and the only ratings last month), recieved no ratings, and remained the most read section, but just barely.

 

• TFSAdmin: How to: Set Up and Manage a Build System recieved the highest (5) rating.
• dv_TeamBuild: both Define Workflow Builds using the Default Template, and How to: Create a Basic Build Definition recieved high (4) ratings.
• dv_TeamVersionControl: How to: View Where and When Changesets have been Merged recieved the highest (5) rating, and How to: Cloak and Uncloak Folders in a Workspace recieved two low (2) ratings.
• dv_TeamProject: How to: Create an Office Project Plan from Team Foundation Work Items recieved this comment: "Good one".

All of the topics that recieved high ratings are either new or significantly reworked for Dev 10. The topic that recieved low ratings is unchanged from previous releases, except for maintenance of links.

 

Define Workflow Builds using the Default Template really only introduces two other topics, and makes the general statement that it will be fleshed out with more info later. I'm guessing that the reader was satisfied with the topics under this one, and not put off that it only contained those two links becaue it is Beta content, and we explained that the topic will be fleshed out later.

 

Watching from Last Month

We saw some potential trends last month. Here's how this month's data relates to that.

• Page Views: Project management, which traditionally gets 1/5 the page views of administration (traditionally the most viewed section). Last month, proportion of page views for most projects remained pretty consistent wiht the traditional distribution, except that dv_TeamProject moved to the top of the stack, with almost 50% more page views than even TFSAdmin. This month, project management is almost exactly the same as TFSadmin, still substantially above it's traditional share.
• Empty Topics: The was no new negative feedback from the empty topics, and there was one high rating for a links-only topic with the disclaimer. It's still clearly too early to know how this practice is being accepted.
• Loc: There's no new data on localization

 

June feedback on Beta 1 content for Team Foundation 2010

Earlier this month, I sent this email internally summarizing the feedback we've recieved on the Beta 1 content for Team Foundation.

Give us feedback

We published help content for Visual Studuio Team System 2010 Beta 1 that is substantially different from what we published in VSTS 2008. We're anxious to hear what worked well for you and what didn't. If you rate the topics that you read, we will use that to understand how to continue to improve the help content. If you have specific comments, please add those. Comments about topics are especially useful, but don't feel like you have to add a comment to rate the topic. Most people, by a vast majority, don't comment or even rate the topics they read. If you do take the time to leave us feedback, you're like to be among only a handful that have done so for that topic, so your feedback will absolutely not be lost in the crowd. I look at the feedback regularly and use that to determine my backlog of work, and that's pretty typical for writers working on VSTS.

Customizing work items

Do you customize the work items that you use with Team Foundation? Kathryn Elliot is looking for your input on what sort of documentation would be most helpful on her blog.

Enabling New Features of Visual Studio Team System 2010 Beta 1 in Upgraded Team Projects

After you upgrade your server to Visual Studio Team System 2010 Beta 1, you have a wealth of new features available. Generally, you don’t have to do anything except start using those new features. For example, you can use the new types of links between work items, such as parent and child links, immediately after the server is upgraded.

In the cases described here, though, you’ll have to do some work if you want to use the new features in team projects that were upgraded from an earlier version of Visual Studio Team System.  You can enable these features immediately after upgrading your server, or you can begin using your team project and enable any or all of them at a more convenient time in the future.

The following posts provide manual steps for enabling each of these features. Hankan Eskici has also provided a script that you may find helpful, especially if you are using MSF for Agile Software Development v4.2 wihtout customizations. Even if you've customized your process, you should be able to tweak the script and use it.

Enable Test Management: Visual Studio Team System 2010 includes features that allow you to manage your tests more effectively.  The screenshots below show the new test case, and the integration of the test steps with a bug. Read Chris Patterson’s blog post to learn how to add test management to team projects that you upgraded to Visual Studio Team System 2010.

Enable Lab Management: You can set up environments to run tests and collect data remotely. For information about enabling lab management, read Vishal Mehrotra’s blog post.

Enable the Agile Planning Workbooks: You can use the new planning workbooks that are included in MSF for Agile Software Development v5.0 to help you plan your project and iterations. If your team project uses MSF for Agile Software Development v4.2, read Aaron Bjork’s blog posts to enable the iteration backlog and product backlog new workbooks.

Enable Branch Visualization: Visual Studio Team System 2010 makes a clear distinction between folders and branches, and adds some new tools to visualize your branch hierarchy and initiate actions such as merges. Read Jamie Cool’s blog post to learn how to convert folders into branches.

Enable Reports: If you open your team project's Reports folder, you'll find that the reports you had been using aren't there. They are still available at http://<server>/reports/<team project>, but they will need to be updated to adapt to the changes to the schema of the cube and the warehouse database. These reports will continue to show the data as it existed before your server was upgraded. Aaron Bjork has a post that explains where the reports can be found and how to move them to the appropriate location on the report server. John Socha-Leialoha has posted information to help you make the changes to your reports so that they work with the new schema and pick up the new data from the upgraded server. Sunder Raman has posted a good article on what those changes are.