Answering some of the questions asked about our recent usability efforts...

Q: Can you tell us how many usability engineers are involved in all those activities?

A: Yes.


Q: How do you like the eye-tracking stuff?

A: It's certainly interesting, and no doubt the "heat maps" look impressive.  What we're going to be focusing on more than anything in the next two eye tracking studies are the effects of the new visual look of the product.  As I've mentioned before, the current visual look that you're seeing is just a temporary skin.  Yet, it's in the product and it's what has formed the baseline of our usability results thus far.

Now, we're to the point where something close to the final visual skin is in our internal builds and, being quite different from the temporary skin, we expect it to have some effect on how people parse, find, and use the new UI.  The next few eye-tracking studies are to learn about the differences between the two skins and figure out if and how we might need to tweak the new look.

(No, the new visual look will not be in Beta 1.)


Q: Will results be made available publicly?

A: Generally, we don't make verbatim Office usability reports available because there are certain internal techniques we use and data we gather that present a competitive advantage to Microsoft.  Our usability team has worked hard to develop innovative methodologies and a rich base of data about how people use Office that helps us compete in the marketplace.

At the same time, you're hearing directly from the product teams of Office for the first time through our blogs, and I'm definitely able to share general results (good and bad.)  I don't have any marketing person telling me what to say or not say, and if I did, you wouldn't read.  I get that.

The other reason you can count on getting a lot of information about the Office 12 UI is that our customers are demanding it.  We realize that no matter how positive the change is, it's still change--and with that comes risk.  My team is committed to providing clear data that shows the benefits and highlights any potential downsides as well (so that we can help people mitigate them during deployment.)

So, I'll be very open about what we learn in general terms, without publishing full 30-page reports online.


Q: What is "Send a Smile?"

A: There's a general philosophy Microsoft has been embracing more and more in all of our beta products, which is that people should be able to send one-off comments as easily as possible, while they're "in the moment."  Windows XP had a "Comments?" link in every dialog box that let you tell us if the dialog was stupid.  Previous versions of Office had the same thing.

Send-a-Smile is a related tool that goes a bit further.  Anywhere, anytime, someone can click a "smiley face" to tell us they like something or a "frowny face" to tell us they don't like something.  We get a lot of context (with the user's permission of course), including a screenshot, sometimes a short movie of the last 30 seconds, related documents, etc.  There's another tool called the Office Feedback Tool (also known as "Ebert") which does a similar thing but with Thumbs Up and Thumbs Down.

All of these tools work on the principal that if someone has to open a newsreader, log onto a newsgroup, type a long message, and send it, we'll lose a lot of valuable feedback just due to complacency.  The idea is to reduce the barrier to entry for sending comments so that we get more data from the "heat of the moment."

And of course, we have all sorts of tools that help us sort an analyze the feedback on the back-end.


Q: OK, really, you better tell us how many people are involved in evaluating the usability of the new Office UI.  Because you blew me off back on question 1.

A: A handful of people are involved full-time, and a lot of people are involved part-time.  Every piece of usability feedback we get from running individual feature tests (in Word, Excel, etc.) is really data about the user interface as well, so it's hard to say exactly.  I will say this: at least as many people have helped with the research and validation around the UI as the core people working on the design of it.  It's a big effort.


Q: But Mr. Wizard, why isn't the volcano doing anything?

A: "I haven't put the vinegar in it yet, Billy.  Hush!"