More Than Just the Two-Way Mirror
I've
revealed in the past how many of the Office 12 UI decisions we make are
influenced by what we learn in collaboration with our usability and research
team.
Personally, I find usability tests scary. You have an idea, discuss it
with others, work it into a prototype and get excited about it. You want
it to work. And then you pull back the curtain, put the idea in front of
real people and all of a sudden you know whether the idea works for real or not.
It's totally humbling.
It's like when you're dieting--you can feel good about the food you eat or
exercise you do, but nothing brings it into focus like stepping on the scale and
facing the result. If the result is bad, it's back to the drawing board.
If the result is good, you set it aside, plan to retest, and start focusing on
where else you can improve.
Over the next weeks, I'm going to write in more detail about our usability and
research efforts around the Office 12 UI. To give you a taste, here's a
snapshot of all the special research projects we have getting started over the next
month or so. The following bullets are excerpted from a status e-mail
message written by Tim
Briggs, one of our usability leads.
- Office 2003 Benchmark (10/19 - 10/20): 30+ people coming through
the Office Design Lab, 10 at a time to complete tasks in previous versions.
We're collecting success/failure, time on task, and satisfaction so that we
can compare to the Office 12 Benchmark later.
- Eye Tracking 1 & 2 (10/19 & 11/11): Using the eye tracker to give
us better understanding of how people use and browse the current visuals so
we can then refine and study the new visuals. We're looking especially
at how people use the chunk titles, the contextual tabs, and the MiniBar.
- Card Sort II (early Nov.): A year ago we had people organize
commands into buckets to help us think about how to generate the tabs.
This time we'll give them our buckets and see how well they can sort the
commands into them, especially while coming up with names for some really
sticky ones.
- Internal Longitudinal Study (Now - Dec.): We're getting people
from the company set to install and start using both current builds and Beta
1. All are outside of Office, most are from non-product groups like
Marketing, Sales, and Legal, and will not only be sending feedback but
participating in the Office 2003 and Office 12 Benchmarks so we can compare.
- The "Truman Show" (Now - Dec.): A brave soul from the "real"
world has agreed to let us take away his Office 2000 and replace it with
Office 12... and do site visits... and work here on campus for a while... and
have conferences via Live Meeting with us... and let us analyze his personal
instrumentation data... and send us a daily journal of his experience.
He's a local guy with a small personal business and excited
about this challenge. As are we.
- Office 12 Benchmark (Nov. - Dec.): We'll look at the whole range
of use from OOBE ("out of box experience") with newcomers to 2 weeks/2
months use with the internal longitudinal people. Like the Office 2003
Benchmark, the tasks are selected to give us some key comparisons we can see
in the big Office Design Lab data and more personally through 1:1 standard
lab sessions.
- Extended Usage Study (Now - RTM): We've started deep engagement
with a large group of users from a local company. We'll be rolling out
Beta 1 to all of these people. As this kicks off, it is the first
major opportunity to closely monitor the rollout, training, adoption, and
acceptance of the new UI over a long period of time. We'll be doing
persistent visits, monitoring instrumentation data, and collecting 1/1
feedback.
- Beta Survey & Visits (Dec. - Feb.): As part of the overall
beta plan, we'll be adding to the benchmark data with a series of site
visits to customers to observe first-hand "use in the wild."
Members of the design team will be partnered up with customers so that we
can follow up more individually starting in December. Well also be
following up with surveys to drill into productivity and satisfaction.
- This is just the known, scheduled research. There are lots of other
efforts going on too like building up the list of instrumentation metrics,
3rd party validation studies, continued "Send a Smile" tracking, continued
consolidation of usability findings from Word, Excel, PowerPoint, Outlook,
and Access, and oh so much more.
Jensen here again. All of this is in addition to the normal ongoing
"behind two-way glass" usability tests that focus on individual mechanisms or
features. And of course it's added to all of the beta feedback we get through newsgroups, customer
visits, e-mail (and even here in blog comments!)
As you can see, we have a lot going on as we strive to learn more and more
about the UI and how to continue improving it. It's a large effort, but I
think it will be worth it.
I'll fill you in on the details of some of the more interesting studies in
the coming weeks.
Have a great weekend everyone!
