I've revealed in the past how many of the Office 12 UI decisions we make are influenced by what we learn in collaboration with our usability and research team.

Personally, I find usability tests scary.  You have an idea, discuss it with others, work it into a prototype and get excited about it.  You want it to work.  And then you pull back the curtain, put the idea in front of real people and all of a sudden you know whether the idea works for real or not.  It's totally humbling.

It's like when you're dieting--you can feel good about the food you eat or exercise you do, but nothing brings it into focus like stepping on the scale and facing the result.  If the result is bad, it's back to the drawing board.  If the result is good, you set it aside, plan to retest, and start focusing on where else you can improve.

Over the next weeks, I'm going to write in more detail about our usability and research efforts around the Office 12 UI.  To give you a taste, here's a snapshot of all the special research projects we have getting started over the next month or so.  The following bullets are excerpted from a status e-mail message written by Tim Briggs, one of our usability leads.

• Office 2003 Benchmark (10/19 - 10/20): 30+ people coming through the Office Design Lab, 10 at a time to complete tasks in previous versions.  We're collecting success/failure, time on task, and satisfaction so that we can compare to the Office 12 Benchmark later.
   
• Eye Tracking 1 & 2 (10/19 & 11/11): Using the eye tracker to give us better understanding of how people use and browse the current visuals so we can then refine and study the new visuals.  We're looking especially at how people use the chunk titles, the contextual tabs, and the MiniBar.
   
• Card Sort II (early Nov.): A year ago we had people organize commands into buckets to help us think about how to generate the tabs.  This time we'll give them our buckets and see how well they can sort the commands into them, especially while coming up with names for some really sticky ones.
   
• Internal Longitudinal Study (Now - Dec.): We're getting people from the company set to install and start using both current builds and Beta 1.  All are outside of Office, most are from non-product groups like Marketing, Sales, and Legal, and will not only be sending feedback but participating in the Office 2003 and Office 12 Benchmarks so we can compare.
   
• The "Truman Show" (Now - Dec.): A brave soul from the "real" world has agreed to let us take away his Office 2000 and replace it with Office 12... and do site visits... and work here on campus for a while... and have conferences via Live Meeting with us... and let us analyze his personal instrumentation data... and send us a daily journal of his experience.  He's a local guy with a small personal business and excited about this challenge.  As are we.
   
• Office 12 Benchmark (Nov. - Dec.): We'll look at the whole range of use from OOBE ("out of box experience") with newcomers to 2 weeks/2 months use with the internal longitudinal people.  Like the Office 2003 Benchmark, the tasks are selected to give us some key comparisons we can see in the big Office Design Lab data and more personally through 1:1 standard lab sessions.
   
• Extended Usage Study (Now - RTM): We've started deep engagement with a large group of users from a local company.  We'll be rolling out Beta 1 to all of these people.  As this kicks off, it is the first major opportunity to closely monitor the rollout, training, adoption, and acceptance of the new UI over a long period of time.  We'll be doing persistent visits, monitoring instrumentation data, and collecting 1/1 feedback.
   
• Beta Survey & Visits (Dec. - Feb.): As part of the overall beta plan, we'll be adding to the benchmark data with a series of site visits to customers to observe first-hand "use in the wild."  Members of the design team will be partnered up with customers so that we can follow up more individually starting in December.  Well also be following up with surveys to drill into productivity and satisfaction.
   
• This is just the known, scheduled research. There are lots of other efforts going on too like building up the list of instrumentation metrics, 3rd party validation studies, continued "Send a Smile" tracking, continued consolidation of usability findings from Word, Excel, PowerPoint, Outlook, and Access, and oh so much more.

Jensen here again.  All of this is in addition to the normal ongoing "behind two-way glass" usability tests that focus on individual mechanisms or features.  And of course it's added to all of the beta feedback we get through newsgroups, customer visits, e-mail (and even here in blog comments!)

As you can see, we have a lot going on as we strive to learn more and more about the UI and how to continue improving it.  It's a large effort, but I think it will be worth it.

I'll fill you in on the details of some of the more interesting studies in the coming weeks.

Have a great weekend everyone!