Designing a new feature is a tricky thing. There are the business decisions, like "Will it help convince somebody to switch to Vista?", or "Will it help save a user money?" There are also technical questions, like, "Can we actually build the thing we have in mind?", and "How much time and writer/dev/test/design/usability resources will it take to ship it on time?" For any feature we make, we need to be confident that its bang to the user is well worth the buck Microsoft spends to develop it.
Those are interesting decisions to make, but in this post, I thought it would be interesting to write about some of the more detailed questions, once we're committed to building it.
We started with a basic goal of the feature: Create help content that can either do the steps for the user, or show the user how to do it himself. That sure sounds like a good idea, but lots of things sound good. How do you know that it's a good idea? One big way to find out if it works is usability testing.
Usability testing is where we get volunteers to come in and try out our software while we watch. (We offer a gratuity, which is usually free software at the Microsoft company store.) We look for different kinds of users, depending on who the product is meant for. For Guided Help, we were testing with beginner computer users; the kind of person who uses Help and would benefit from a tool to help them get some multi-step task done. Usability testing is usually done an early version of the software.
For some of our early Guided Help usability tests, we used animated mock-ups, or even just Powerpoint slides, instead of running code. We do that because we need to figure out the right way to build something before we invest time and effort in making the real code.
With a prototype, we can have two or three different designs to try out, and see what works better.
Some major lessons that usability testing taught us:
We need both Do It and Show Me
We tried often to just pick one and go with it: after all, simpler is better, so if we could eliminate a choice, that would be good. But it turns out that different users prefer different modes, and it always came out about 50-50.
We also thought Do It mode people would be those who were into efficiency, and just wanted to get something done. And we thought that Show Me fan would be beginners who wanted to learn.
In fact, what we discovered is that the people who like Show Me best are those who like to see details. There's a personality type that says, "I'd rather watch each step" that doesn't seem to have much to do with how much experience you have with computers. And Do It mode folks were those who were comfortable with just handing over control to the computer, and didn't even want to know the details. We affectionately nicknamed the two main personality types the "micromanagers" and the "delegators". :)
Do It mode shouldn't be too fast
In our original designs, Do It mode would just zap through each step, and the UI would go by in a blur. Every single user got confused by that. Even though they just clicked a button saying "Do it for me", when it actually happened in front of their eyes, they didn't like it. So we slowed it down, and added an animated green arrow to look like the mouse pointer going around clicking.
Lead the user's eye where to go next
There are points where Guided Help will highlight a control, such as a listbox, and let the user pick something. When the user's done picking, the user has to click Next in the Guided Help window. In usability, we noticed that most people didn't realize that. They'd choose something in the list, and then stop, confused about what to do next. So we added bubbles to draw the users' eye to the Next button. 
Wording
The wording of the links to "Do it automatically" or "Show me step by step" is very hard to get right. The problems are 1) Guided Help is a new thing, so nobody knows what to expect if you click the link, meaning we have to let them read about it 2) People don't read. So if we do explain it, it doesn't matter because no user will read 8-10 words if they can help it; we all prefer to scan for keywords we recognize and click on the first one that looks right. So Dave Johnson, the writer for Guided Help (who also had a significant hand in lots of the design decisions), worked on a lot of revisions of the text to make it brief, but understandable. We learned through usability testing that the link with gray sub-text works okay: users are willing to read the link because it's so very brief (just "Do it automatically").
They can see by the way the sub-text is gray that it's optional text. It's only when they realize that "Do it automatically" doesn't quite tell enough that they go ahead and read the sub-text. Believe it or not, this kind of quick micro-decision about where to put our attention really happens. That's how we all read pages. That's why newspapers and web-pages have columns (a narrow block of text is less intimidating), and organizes the page with headlines and sub-titles, so you can scan quickly to decide what to read.
Ignore the giant icon!
Even with the big icon, and the two links at the top of the help topics, many people would completely ignore them. This blew our minds at first. A user opens a help topic, and there, right at the top, with a big juicy graphic, are words "Do it automatically" or "Show me step by step". What do users' eyes do? The zap directly to step 1 of the help topic. Something about the size and the way the graphic plus links are a unit makes them 'ignorable', like a banner ad at the top of a webpage. The solution? Change step 1 so that it reads "Click one of the links above…" You can see that special step 1 in the first screenshot, at the top of the page.
We learned a lot from usability, and we'll learn even more when people starting using it for real. This is a 1.0 feature for us, so we know we'll have made some assumptions that turn out not to be true. We've done all we can do to check those assumptions, but learning from our users never stops.
Andrew McGlinchey
Program Manager, Guided Help
