The Social Platform, as well as the other MSDN & TechNet apps, is hosted by the Microsoft.com Operations team, with servers in the same data centers that host Live, MSNBC, etc.
I've been fascinated to learn how much application performance varies based on a user's proximity to our data centers. The Social Platform is primarily hosted in the Columbia data center, near Quincy, WA, and a Blue Ridge, VA facility.
As you might expect, performance is really good for users near those data centers. And it's also good in locations that have fat pipes connected back to the areas. For example, perf is great in San Jose and Vancouver, WA.
But until we added CDN support to the Social Platform, performance at more remote locations was poor. For example, our pages were noticably slower to load in the Southern US, and took twice as long to load in parts of of Europe and three times as long in China.
Now that we've added CDN support through Akamai and ChinaCache, performance is stunningly better. Performance is pretty consistent throughout the US and Europe, and while it's still slower in China, it's less than half the time it was before.
By the way, "CDN" stands for Content Delivery Network. The way it works is that we ask the CDNs to cache static resources, such as images and JavaScript files, on servers that are distributed throughout the world. The CDN companies use DNS and routing tricks to serve the cached files from servers that are close to end users.
It turns out that most of the extra time required for a page to load in Europe or Asia is caused by network latency. It can take many tenths of a second, or even several seconds, for a request to make its way over the the network and back from the server. This problem is worse when you're connecting to servers overseas because you have to traverse more nodes on the network.
Another critical factor in page load time is that IE and other browsers only open two connections at a time to any single server. If there are lots of images and scripts on a page, there will be additional network latency for each pair of resources that are requested.
We work around that limitation by hosting the files for any given page on up to five different host names. While the files are physically on the same server, we split URLs across five different hosts to allow parallel loading.
When people learn that I work on MSDN and TechNet, one of the first things I hear is that our Search solution is really terrible. The best way to find something on our sites, they say, is to use Google.
A year ago I would have agreed, but now I’m proud to be able to tell people that we’ve been updating Search over the past year, and they should really take another look.
I had a fun experience last night that illustrates just how far we have come. Tim and Drew, developer friends from Brisbane, Australia are in town and came over to my place to hang out. We yakked about all the normal stuff—families, politics, American accents—but as web guys, we ended up talking shop a lot.
I proudly demoed our new forums and bookmarks applications, and they were duly impressed, but one of them said, “You know what I really can’t stand about MSDN? It’s the search—it’s just terrible. I just use Google.”
Well, this was a great opportunity, so I said, “When’s the last time you used MSDN search?” They both said they had given up on it a year or two ago, so I said, “Check this out…” and proceeded to show off our new Search app, focusing especially on refinements and auto-complete.
“Wow, that’s pretty interesting,” said Drew, “but let me see how it works on something I really care about—try search for the Win32 CreateWindow function—I use it all the time.” I did as he asked, typing only “create” into the search box before “createwindow” showed up as the fourth item, then a “w” so it was first. Selecting it and pressing enter, Drew was pleased to see CreateWindow() show up as the first item in the results list. He also like that there was a “Win32” refinement so he could narrow the results to Win32 content only.
“This is great.” Growing ambitious, Tim added, “What really pisses me off about Google is how it strips out all punctuation, which is vital when searching for programming terms.” He asked me to search for the XmlDocument.CreateElement method. “It would be really great if you search supports periods.”
I typed “xmldocument” and Timsaw a bunch of autocomplete terms with “dots” in them. “This looks promising--I'm ecstatic,” he said. I typed a couple more characters to get “xmldocument.c” and “xmldocument.createelement” was listed as the second item. I selected it and the results came back with all the different versions XmlDocument.CreateElement() listed first.
“This is absolutely brilliant,” said Tim. “This is exactly the stuff I was searching for today. I’ll definitely start using MSDN search again.”
For someone who five minutes earlier was saying that MSDN search is not any good, this was a great transformation to see.
I've been meaning to blog more often, and some of the questions I'm getting about our new Social Bookmarks app have given me a good excuse to talk about the thinking behind the Social Platform.
This blog post started with some tough questions from a senior and well-respected guy within Microsoft who saw our new Social Bookmarks app for the first time today. He asked, "Who needs another bookmarking app? And why did we build a new bookmarking solution for MSDN and TechNet rather than Windows Live Favorites or Sharepoint?"
The answer is actually more about tagging than it is bookmarking. The two concepts are used almost interchangably--every social bookmarking site includes tagging, and most tagging applications are fundamentally about bookmarking.
But for the Social Platform, tagging is about much more than bookmarking.
Tagging is deeply integrated into our platform strategy, and we're using our centralized tagging and feeds service (called Discovery Services) internally to power all sorts of interesting scenarios for MSDN, TechNet and Expression. Indeed, tagging is so central to our strategy that we would be building our central tagging service even if we chose not to build a Social Bookmarks app.
For example, take the feeds that are everywhere on our sites. (Note that I didn't say "RSS Feed"--yes, all our feeds are available via RSS, but feeds are not always RSS and they're useful with or without RSS.) For example, consider the C# headlines feed on the C# DevCenter. To make a news item show up in this feed, our site managers simply define the item in Discovery Services with a title, description and URL (and some other metadata) and tag it with C#. That's it. The feed is a dynamic query for all items for that our site managers have tagged with C#.
But the real power is that every item can have multiple tags. Take, for instance, the top headline (as of tonight) on the C# DevCenter: "Microsoft .NET Framework 3.5 Service Pack (SP) 1 Beta Now Available". To include this same item in the .NET Framework headlines feed, our site managers need simply tag it with .NET too. This is way easier than the process they've had to follow to date, which is manually placing the item in both feeds. The new approach saves lots of work for site managers.
MSDN, TechNet and Expression are built around Centers and applications like the Library, Forums, Code Gallery, Blogs and Bookmarking. Centers are portal pages that help users to find their way around our sites. All of the content shown in a portal page are either feeds or fragments of manually authored content. And every feed and every fragment can be--you guessed it--tagged.
In addition, our platform has the idea of "related links," which is an ordered list of links that we (that is, Microsoft) define for a tag. Using this idea, we can suggest links related to a currently selected tag. For example, we would likely define C# DevCenter, C# Library, C# Forums, C# Blogs and C# Downloads as related links for the C# tag.
If you're still with me, you may be getting an idea of where I'm going: We can automatically generate portal pages using tags.
It's really expensive to manually create portal pages, so we only produce a few hundred Centers dedicated to Microsoft products, technologies and other important topics like security. So automating creation of these Centers would be a good thing because it would free up valuable people to add value in other ways.
But it gets better: While it's nice to automate something that's currently manual, the real value is that we can automatically create a portal for ANY tag. So you want to see the portal for the Ajax tag? How about the Exchange Virus Scanning API (VSAPI)? No problem...just click either of the Ajax and VSAPI tags.
Cool, eh?
Now that I've talked about how we're using tags to create dynamic feeds and portals, I should return to Social Bookmarking.
I mentioned above how our site managers create feeds simply by tagging items in Discovery Services. It turns out the exact same thing is true for Bookmarks, only users are doing the tagging rather than Microsoft employees. The added bonus is that there are millions of users and Bookmarks are Social, which means that we can infer popularity by how many users bookmark and tag a given item, and by how often a particular tag is applied to one or more items.
Our goal is to get tens of thousands of users to bookmark and tag content on our sites and elsewhere on the Internet. (To put this in perspective, there are tens of millions of visits to MSDN and TechNet every month.) Every bookmark and tag provides information that can help other users, and we can use the collective wisdom of many users to surface the content most likely to be interesting to users interested in a particular topic.
While our site managers create Microsoft-recommended feeds for official announcments and other topics we think our users will find interesting, Social Bookmarks leads to social tag clouds and feeds that are democratically organized by the community itself. My hope is that put into this context you see that Social Bookmarks represents the social aspect of our overall tagging strategy.
And this explains why we've built our own Social Bookmarks application: It's not just about bookmarking.
The blogosphere is starting to buzz about our new Social Bookmarks preview app that went live today. While most bloggers are focusing on the MSDN version, we have also shipped versions for TechNet, Expression (screenshot below) and a generic Microsoft brand.
While it's true that our Social Bookmarks app provides features similar to those found in del.icio.us and our earlier TagSpace prototype, our approach is a little different. While our bookmarking app can be used to bookmark anything by anyone, my team is really focused on technical professionals--developers, system administrators, designers and other technical folks. In fact, we've moved away from the "TagSpace" name because we're not trying to introduce a new brand name. Instead, we are focused on making our MSDN, TechNet and Expression as social as possible.
Given our focus on making our sites more social, there are a couple big concepts that are unique to how we're doing bookmarks:
First, Bookmarks are brand-specific. In other words, MSDN bookmarks are kept separate from TechNet bookmarks. The idea is that developers care about different topics than IT professionals, so it's useful to keep the bookmarks separated. For example, a system administrator probably isn't interested in seeing a bunch of bookmarks related to .NET APIs.
When you view My Bookmarks, you see everything you've bookmarked across all brands. But when you view All Bookmarks, you'll just see only the bookmarks that apply to the current brand. For example, if you bookmark something in an Expression forum, you won't see it listed under MSDN's All Bookmarks view.
The next new concept is that tagging forums threads is an alternate way of bookmarking. In fact, the tag control in the forums app is now labeled "Bookmarking" and it provides a link to your bookmarks as well as letting you tag the current thread.
We call this "inline tagging" and plan to roll it out more widely across the sites. Later this year, bookmarks in the MSDN Code Gallery, and the MSDN and TechNet libraries will use similar inline tagging controls, and they will share our new Social Bookmarks backend.
I'm looking forward to see how MSDN, TechNet and Expression users respond to our new bookmarking capabilities. My hope is that they will appreciate the convenience of bookmarking on our sites, while also providing lots of tags that will help other users find topics that are helpful and interesting.
There are several interesting things coming soon from the Community Platform team.
Social Bookmarking
We're releasing a nice Social Bookmarking Preview in late May. Our marketing team is planning some minor hoopla to show the app off at TechEd.
MSDN, TechNet and Expression will all have their own instance of the bookmarking app, so you will experience the app as MSDN Social Bookmarking, TechNet Social Bookmarking, etc. Nevertheless, you are not constrained to bookmarking items on these sites, or even within Microsoft. You will be able to bookmark almost any item on the Internet that has a URL (the exceptions being adult sites and the like).
Social Bookmarking is a useful way for you to save your favorites on the server and access them from any machine or browser. This in itself is helpful for technical professionals who frequently move between machines and browsers. You can apply one or multiple tags to each item you bookmark to help you browse and search your bookmarks later.
The Preview will show off some interesting ideas about how bookmarks and tags can be socially shared. For example, you'll be able to view an aggregated list of bookmarks from all users, along with an aggregated tag cloud. This is an interesting way to explore what other users are interested in, and to discover topics that are popular now. (And yes, we do support private bookmarks that only you can see!)
Most of the core functionality that we're delivering now has been done by other sites like Del.icio.us. What's new is that our experience is fully integrated into the Community Platform and will therefore be available to a wide range of users who have not used other bookmarking sites. In addition, we're moving toward a fully integrated tagging and bookmarking strategy where Forums, CodePlex and Library Annontation (aka Wiki) tags are combined so they can be browsed in a common tag cloud and managed in one place. (Only Forums tags will be integrated in the Social Bookmarking Preview.)
Updated Profile
We're enhancing User Profile to provide a richer experience to view information about yourself and others. Our May release will feature an updated UI that lists the forums contributions and bookmarks for every user. When you view your own profile, you'll have the option of editing your settings.
These changes make each user's profile more attractive and interesting, but they are just a small step in our larger plan to make Profile a rich, central place for users to present themselves to the community and manage their stuff within MSDN, TechNet and Expression.
Forums Migration
While we continue to improve lots of things with Forums 3.x, an all-new Forums solution, the big news is that we're preparing to migrate the old forums to the new application. This migration will start with about 60 forums that we plan to migrate by the end of the month. Examples include the Forefront and Visual Studio forums.
We are trying to be really careful to minimize any disruption for users, moderators and product teams who depend on the forums every day. The migration is meant to be robust, so we are migrating forums posts and threads, user info and user points. One of the thornier problems is that there are 10's of thousands of existing links to the forums, so rather than letting those links break when we move the forums, we're going to update the existing forums to correctly redirect incoming requests for migrated threads to the migrated version of the thread.
We currently expect to migrate the forums in chunks over the summer. The migration process is fairly automated, so it would not be technically challenging to migrate them all at once, but our users and moderators have asked to move more slowly so they have time to prepare for the change.
I've got a really interesting job opening for an experienced technical PM who wants to work on some fun Web 2.0 stuff. Here's the detail...
Job Description
DevDiv’s Server & Tools Online team is seeking a highly technical senior program manager to help us build innovative new Community Platform web applications, including forums, community-driven portals, blogs, social bookmarking, a feeds and tags service, plus content management. Our team is focused on transforming MSDN, TechNet and the (new) Expression web sites into the most innovative, community-focused web sites within Microsoft and the industry.
We are seeking a well-rounded senior PM with excellent technical, design and leadership skills. You are a good fit if you have strong passion and experience building great web apps, if you are good at defining and analyzing problems and if you know how to get things done.
This job is a good fit if you…
- Have great experience as a PM working on large-scale web applications
- Are experienced with web architecture and performance
- Understand the social web (e.g. social networking, online communities, web 2.0, etc.)
- Are passionate about building great software
- Possess great abstract thinking and analytical skills
- Have deep understanding of complex web technologies and sites
- Care about the details
Specific qualifications include:
- Strong technical background with expertise in web technologies like HTTP, ASP.NET, SQL Server, AJAX, JavaScript, XML. LAMP experience also welcome!
- Work experience directly related to engineering of complex web sites, tools and user experiences
5+ years successful experience as a technical PM. (Microsoft PM experience desirable but not required.)
- Strong cross-group collaboration skills
- Demonstrated ability to write stories, functional specs and other technical documents
- Experience shipping products/web sites
- BS or higher in technical field
The first version of the STO Community Platform went live on Valentine's Day, Thursday, February 14.
My team has built this platform to host all community applications for MSDN, TechNet and the new Expression site. This release includes an all-new Forums app, a new Profile app, and a rewritten Search app.
The most prominent use of the new Forums app is the Expression Forums site. MSDN and TechNet forums are mostly still running on our older "Forums 2.4" codebase. The Community Platform Forums for TechNet and MSDN are live but primarily with feedback and sandbox forums only for now.
Migrating existing 2.4 content and users to the Community Platform is a top priority for our team over the next 100 days. Our goal is to have a substantial chunk of MSDN forums migrated by the end of May in support of a “re-launch” of MSDN & TechNet at TechEd in early June.
New Applications
This Community Platform release features many new applications:
· Forums 3.0: 100% new code that will replace the existing Forums 2.x solution in coming months. Highlights include a phenomenal new UI, automatic updates and our first social tagging deployment. Users and stakeholders who have seen pre-release versions of forums are raving about its rich capabilities and user interface.
· Search 3.0: A rewrite of the existing MSDN/TN search application with improved auto-complete functionality, improved international search (including 20 languages!), and dramatically improved refinements.
· Profile 1.0: A new centralized place for users to manage their personal settings, such as display name, avatar and signature.
· Discovery Services 1.0: A high performance feed and tag “engine” that provides a RESTful interface to query editorially managed and tag-driven dynamic feeds. Forums tagging, search refinements, RSS feeds, forum categories and related sites are some of the features enabled by Discovery Services.
· Feed Management Tool 1.0: A browser-based internal tool that lets site managers and other authorized stakeholders create, edit, tag and schedule both feed items and feeds that are available in both RSS 2.0 and ATOM formats.
· STO Apps 1.0: A centralized portal and framework for all Community Platform management tools. The feed management tool, forums admin, global profanity filter management, tag management and all other Platform tools are available on this site and make use of a shared framework that provides consistent UI and role-based security.
Platform Features
In addition to the stand-alone applications listed above, there are many foundational capabilities and features in the Community Platform. For example:
· Shared UX: The framework is highlighted by the all-new Coronado/Boston masthead design that will be rolled out in all MSDN, TechNet and Expression apps, including the Community Platform apps, the Library, Centers, Code Gallery and future Platform apps.
· Social Tagging and Feeds: Every application can leverage the rich social tagging and feed support enabled by Discovery Services.
· Feed Service End Point: A universal endpoint to query feeds from any Community Platform application.
· Friendly URLs: Simple URLs that are easier to type and remember.
· Globalization Framework: Rich standardized globalization support that can be extended and customized as needed by individual apps.
· Multi-Brand Support: Every application is available in each brand. MSDN, TechNet, Expression and Microsoft brands are supported currently.
· Shared App Framework: All apps are built on a common framework that provides master pages and controls, config, instrumentation, error logging and much more.
What’s Next
This release represents the completion of my teams's Coronado project, and we're now moving onto the next wave of Community Platform development, code named Catalina. In addition to updates to all existing applications and the Platform itself, Catalina will feature a new Portals application, Content Management, and a Social Tagging and Bookmarking solution. More details to follow soon!
Welcome to my new blog, where I plan to write about my team's work to enhance the MSDN and TechNet web sites.
I am the Group Program Manager for the Server & Tools Online team. I am one of the managers for a standard Microsoft engineering team that consists of program managers, developers and testers. Roughly speaking, program managers work with customers and other stakeholders, like product groups or customer support, inside Microsoft to figure out and prioritize business requirements. We also work with developers and testers to define the functionality and features that are built.
There are several engineering teams in my larger business group. A large team builds the MSDN & TechNet Publishing System (MTPS), which is especially good at hosting the MSDN and TechNet libraries and the MSDN Wiki (for a wiki example, see Community Content on library pages). Another team builds Codeplex. We have yet other teams who are responsible for subscriptions and downloads.
My team is responsible for other community content on MSDN and TechNet, including forums and blogs. We also build the new MSDN and TechNet search application, which provides dramatically improved search for the sites compared to earlier solutions. Our team also built the Learning Platform, which helps MSDN and TechNet users find and acquire learning resources, such as eLearning courses and books.
My team is developing a new Community Platform solution, which provides a common architecture and a suite of services that will enable the next generation of MSDN and TechNet applications. We are currently building an all-new forums application (first version available in early 2008) on top of the community platform, and we have ported the search application to it. Later in 2008, we plan to release portals, blogs and social bookmarking applications on the Community platform.
The very name of our new platform reflects the degree to which we plan to embrace and enable Community applications for MSDN and TechNet. As a shorthand, we talk about enabling Web 2.0, by which we mean allowing our users to both consume and contribute content to our sites.
I plan to use my blog to talk more about my team's vision for MSDN and TechNet, the Community Platform, and the applications we're writing for the platform. I hope someone finds it interesting and helpful.
