Usability Redux
Answering some of the questions asked about
our recent usability efforts...
Q: Can you tell us how many usability engineers are involved in all those
activities?
A: Yes.
Q: How do you like the eye-tracking stuff?
A: It's certainly interesting, and no doubt the "heat maps" look impressive.
What we're going to be focusing on more than anything in the next two eye
tracking studies are the effects of the new visual look of the product. As
I've
mentioned before, the current visual look that you're seeing is just a
temporary skin. Yet, it's in the product and it's what has formed the
baseline of our usability results thus far.
Now, we're to the point where something close to the final visual skin is in our
internal builds and, being quite different from the temporary skin, we expect it
to have some effect on how people parse, find, and use the new UI. The
next few eye-tracking studies are to learn about the differences between the two
skins and figure out if and how we might need to tweak the new look.
(No, the new visual look will not be in Beta 1.)
Q: Will results be made available publicly?
A: Generally, we don't make verbatim Office usability reports available because
there are certain internal techniques we use and data we gather that present a
competitive advantage to Microsoft. Our usability team has worked hard to develop
innovative methodologies and a rich base of data about how people use Office
that helps us compete in the marketplace.
At the same time, you're hearing directly from the product teams of Office for
the first time through our blogs, and I'm definitely able to share general
results (good and bad.)
I don't have any marketing person telling me what to say or not say, and if I
did, you wouldn't read. I get that.
The other reason you can count on getting a lot of information about the Office
12 UI is that our customers are demanding it. We realize that no matter
how positive the change is, it's still change--and with that comes risk.
My team is committed to providing clear data that shows the benefits and
highlights any
potential downsides as well (so that we can help people mitigate them during
deployment.)
So, I'll be very open about what we learn in general terms, without publishing
full 30-page reports online.
Q: What is "Send a Smile?"
A: There's a general philosophy Microsoft has been embracing more and more in
all of our beta products, which is that people should be able to send one-off
comments as easily as possible, while they're "in the moment." Windows XP
had a "Comments?" link in every dialog box that let you tell us if the dialog
was stupid. Previous versions of Office had the same thing.
Send-a-Smile is a related tool that goes a bit further. Anywhere, anytime,
someone can click a "smiley face" to tell us they like something or a "frowny
face" to tell us they don't like something. We get a lot of context (with
the user's permission of course), including a screenshot, sometimes a short
movie of the last 30 seconds, related documents, etc. There's another tool
called the Office Feedback Tool (also known as "Ebert") which does a similar
thing but with Thumbs Up and Thumbs Down.
All of these tools work on the principal that if someone has to open a
newsreader, log onto a
newsgroup, type a long message, and send it, we'll lose a lot of valuable
feedback just due to complacency. The idea is to reduce the barrier to entry for sending comments
so that we get more data from the "heat of the moment."
And of course, we have all sorts of tools that help us sort an analyze the
feedback on the back-end.
Q: OK, really, you better tell us how many people are involved in evaluating
the usability of the new Office UI. Because you blew me off back on
question 1.
A: A handful of people are involved full-time, and a lot of people are
involved part-time. Every piece of usability feedback we get from running
individual feature tests (in Word, Excel, etc.) is really data about the user
interface as well, so it's hard to say exactly. I will say this: at least
as many people have helped with the research and validation around the UI as the
core people working on the design of it. It's a big effort.
Q: But Mr. Wizard, why isn't the volcano doing anything?
A: "I haven't put the vinegar in it yet, Billy. Hush!"
