Jensen Harris: An Office User Interface Blog : Research and Evaluation

The Story of the Ribbon

jensenh — Wed, 12 Mar 2008 10:00:00 GMT

I was reading through commentary from people who attended last week's MIX conference in Las Vegas. Running across Miguel de Icaza's kind words reminded me that I hadn't posted a follow-up about my MIX talk yet.

Last week, I presented a session at MIX called "The Story of the Ribbon." I talked a bit about the general design process we used to come up with the Office 2007 user interface, to iterate on it, and to evaluate it. As part of the discussion, I showed for the first time some of the early prototypes we worked on (and abandoned or refined) along the way.

It's always fun to present substantially new content, and this was my first time giving large portions of this talk. The audience was great and, although you can't hear them on the video, they seemed to be into it and enjoying the presentation. It was a lot of fun!

Watch "The Story of the Ribbon"
(Video, audio, and slides)

Download "The Story of the Ribbon"
(Slides and audio only, Windows Media, 146 MB)

Alternate Formats:

Download for iPod
(.mp4, 121 MB)

Download the PowerPoint slides only
(.pptx, 20 MB)

Dowload the slides only as a PDF
(.pdf, 19 MB)

Although I showed a few prototypes, I truly only scratched the surface of what the team created during the design phase of Office 2007. I spent a weekend painstakingly going through thousands of pictures to choose a few representative samples to show. Because I only had 75 minutes, I knew clicking through 25,000 pictures probably wasn't going to work.

Here are photos of the beginning and the end of the talk courtesy of Long Zheng. (You'll have to watch the presentation to see what's in-between!)

Over the last few days, the screenshots of the evolution of Word from version 1.0 to 2003 have been lifted from this presentation and subsequently posted and reposted all over the web.

That's OK, but if you want to see the full, original screenshots along with the commentary and discussion, please read parts 2, 3, and 4 of the Why the UI? series of posts.

While at MIX, I also participated in a panel discussion called "What's the Secret Formula?" along with Mike Schroepfer from Mozilla, Dan Harrelson from Adaptive Path, and Daniel Makoski from the Surface team at Microsoft. This was an interesting discussion about some of the challenges inherent in delivering on great user experiences.

Watch "What's the Secret Formula?"

Thanks to everyone who came up and introduced themselves after the session and throughout MIX. I enjoyed talking to you and meeting so many of you face-to-face!

Computers Can Do That? (Real People Study, Part 4)

jensenh — Thu, 13 Jul 2006 10:00:00 GMT

Today's Guest Writer: Rich Grutzmacher

Rich Grutzmacher is a Program Manager on the Office User Experience team. He helped coordinate one of the long-term, real-world studies conducted on early Office 2007 builds.

This is the fourth in a series of entries where I will share with you some of the lessons we learned by following a group of typical Office users for eight months while they used Office 2007 Beta 1 to perform their everyday work.

Last time, I discussed how we used information gathered through extended observation to make changes to the product that address usability issues we normally wouldn't have found until we shipped the product and would have had to put off fixing until the next version. Today, I would like to share with you how we used the extended usage study to determine whether or not we achieved our design goals for the Office 2007 UI.

One of the primary design goals for Office 2007 UI was to help people discover more of the features that would help them get work done more easily, but that they simply can't find or don't realize exist in Office.

You may recall from reading Jensen's blog that roughly half of the Top 10 most-frequently requested features in Word already exist in the product. There is nothing more embarrassing to a Microsoftie than that Homer Simpson Moment when a friend or family member says, "You mean I can do that in <Insert Your Favorite Microsoft Software Here>?"

In our extended usage study, we found that nearly all participants stated without prompting that the user interface in Office 2007 exposed them to more of the depth of the product. One participant in the study went as far as to state, "Office 2007 helps me find stuff I'm not looking for."

All of the customers in the study commented that they were using features that they did not use in previous version of Office because they were now "more visible and easier to get to."

Customers reported finding the user interface as "more browsable" and "enjoyable to browse." One participant even said, "it's more fun to search when you can't find something." (I personally found that one hard to believe until I saw the video footage for myself.)

Given our deep understanding of how each participant used Office 2003, we were able to verify their statements about Office 2007 by having them show us how they are using Office differently.

An example of this finding was best demonstrated by two participants who never used Conditional Formatting in any of the previous versions of Excel.

With Office 2007, they quickly discovered Conditional Formatting and began using it regularly to perform their everyday work tasks. Over time, they came to depend on Conditional Formatting to do their work. Neither participant realized that Conditional Formatting had been in Excel for the past several releases of Office.

What was most striking about these conversations was how positive people felt about browsing to find new features. They stated that they could easily find and use features they needed most commonly, and then when they needed to use less-common features, they knew how to do so. The fact that the participants were continually discovering helpful features they didn't previously know about was just icing on the cake.

In the end, this study helped us validate that we achieved one of our primary design goals for the Office 2007 UI. The new user interface helped make many features in Office more discoverable and, as a result, people were able to use more of the power of Office help them perform their job.

Putting the Feedback to Work (Real People Study, Part 3)

jensenh — Tue, 11 Jul 2006 17:00:00 GMT

Today's Guest Writer: Rich Grutzmacher

Rich Grutzmacher is a Program Manager on the Office User Experience team. He helped coordinate one of the long-term, real-world studies conducted on early Office 2007 builds.

This is the third in a series of entries where I will share with you some of the lessons we learned by following a group of typical Office users for eight months while they used Office 2007 Beta 1 to perform their everyday work.

Last time, I discussed how exciting and encouraging it was to verify for the first time with users outside of Microsoft that they could quickly learn how to use Office 2007 to do their own work with virtually no training. Today I would like to share with you an anecdote that demonstrates how feedback obtained from participants using Office 2007 Beta 1 for an extended period of time was used to improve the product before Beta 2 was released to the public.

Those who frequently read Jensen’s blog will likely agree that designing the user interface for an application like Office can be tricky. Often, our most difficult design decisions involve how to make features both discoverable and efficient to use. Just imagine how annoying it would if every time you clicked on the Print icon a little balloon pop-up saying, “Did you know that you can use the print settings dialog to change from single-sided printing to double-sided printing!” I am sure that many people would find this unacceptable, but it would certainly score highly on the discoverability scale.

As designs that optimize for efficiency sometimes require a bit of learning, we were not only required to evaluate if new features in the Office 2007 UI were discoverable, but also if they were learnable. To effectively answer this question, long-term observation of participants using Office 2007 Beta 1 was required.

A prime example of behavior that required learning in involved locating the View commands. In Beta 1, the View Menu was located on the status bar in the bottom-right corner of the screen.

In Beta 1, View features were in a menu at the bottom of the window

We knew from our earliest usability studies that the View menu was not particularly discoverable with this design, but we believed that it would be easily learnable and would provide efficient access to the View commands. This design also fit nicely with the zoom and document view buttons placed on the enhanced status bar in Office 2007.

Unfortunately, our belief that the location of the View commands on the status bar would be learnable turned out not to be true. This was demonstrated best by one study participant in the extended usage study who sent us a Smile during her first couple days using Office 2007 to tell us how much she liked the new View Menu on the status bar.

Two weeks later, this same study participant sent us a Frown to tell us how frustrated she was because she could not find Split Window (a command in the View Menu).

This type of feedback concerning the learnability of Office 2007 could only be obtained through long-term, extended observations of people using the product to do real work. In this case, these observations played an important role in helping us make the decision to create a View tab for the Ribbon, which appeared in Beta 2.

This change greatly improved the usability of the product, because it matched people's expectations that all features were found in the Ribbon. When we did follow-up visits with newer builds, we found that all of the previous problems accessing View functionality had simply melted away.

We found and fixed literally dozens of issues like this directly based on feedback from the extended-usage studies--issues we normally wouldn't have found until we shipped and would have had to put off fixing until the next version.

You Mean I Don't Need To Retrain Everybody? (Real People Study, Part 2)

jensenh — Thu, 06 Jul 2006 17:00:00 GMT

Today's Guest Writer: Rich Grutzmacher

Rich Grutzmacher is a Program Manager on the Office User Experience team. He helped coordinate one of the long-term, real-world studies conducted on early Office 2007 builds.

This is the second in a series of entries where I will share with you some of the lessons we learned by following a group of typical Office users for eight months while they used Office 2007 Beta 1 to perform their everyday work.

Yesterday, I shared with you the rationale behind the study. Today, I will share with you a few of our early observations related to training (or the lack thereof) that we obtained shortly after the release of Office 2007 Beta 1 in 2005.

One of the most common questions people have asked is "how much training will it require to effectively use Office 2007?"

While we are providing many tools to help with the transition, many concerns can be allayed by simply trying out the product. In our experience, once they try it for themselves, most people agree that extensive training is not required to use Office 2007, despite the new user interface.

As Jensen mentioned a few weeks ago, we designed the new user interface with the goal of requiring little training to use Office 2007. However, we also knew that our users would be the final judges and that we needed to get the UI in front of typical Office users as early in the process as possible to see what we could learn about the learning curve of the product.

One of the primary research questions for this extended usage study was: "Can typical Office users quickly learn how to use Office 2007 with virtually no training in their own natural work environment while performing their own daily work tasks?"

To help answer this question, we tracked the participants very closely for the first 2 to 3 days they had Office 2007 Beta 1 installed. (I recall driving in the snow to visit with participants. Wow, I can't believe it was that long ago!)

During these early interviews, we found that all participants were able to accomplish their primary work tasks during their first 48 to 72 hours using Office 2007. No test participant encountered any work stoppage issues and no one reported any subjective decline in their performance or in their ability to perform any work-related tasks.

In fact, the participants' out-of-box experience was overwhelmingly positive.

Participants uniformly reported liking the new interface, but felt that it would take some time to get fully used to. Nearly all of the issues encountered during the first 48 to 72 hours were related to the beta quality of the software and were not a result of the fundamental changes to the user interface.

This finding was very encouraging since it was the first time anyone outside of Microsoft was using Office 2007 to perform meaningful work tasks in a natural work environment. These results helped validate our hypothesis that users could quickly and easily learn how to use Office 2007 even without the benefit of training.

With virtually anything in life, a little bit of training can help you be more effective. That is undoubtedly true with the Office 2007 user interface (and would have been true with the previous user interface as well.) But this extended usage study helped us have the confidence that even on day one, people could get their work done--even without the benefit of training.

As we move towards availability of the final product, case studies which help demonstrate these results will become available for us to share with you.

Next time: How we used feedback from the study to improve the product for Beta 2.

Real People Doing Real Work with Office 2007 (Part 1)

jensenh — Wed, 05 Jul 2006 17:00:00 GMT

Today's Guest Writer: Rich Grutzmacher

Today I present guest writer Rich Grutzmacher, Program Manager on the Office User Experience team. He helped coordinate one of the long-term, real-world studies conducted on early Office 2007 builds.

For eight months, we followed the progress of a group of twelve people in a Fortune 500 company here in the Pacific Northwest who used Office 2007 to perform their daily work. These people graciously agreed to install Office 2007 Beta 1 on their main work computers and to let us track their feedback and thoughts over a long period of time using the software.

The participants in the study were not software engineers or in any way associated with high-tech; they were just normal people who use Office to help get their jobs done.

During this extended usage study, we personally interviewed and videotaped each participant every 2 to 3 weeks to understand how they were using Office 2007 to perform their everyday work tasks. The participants in the study also sent us feedback using Send a Smile whenever they encountered a problem or discovered something about the product that they liked.

Given the issues one encounters while using early beta software, we considered nominating these participants for Sainthood. Their willingness to endure the pain of using a beta product for an extended period of time to perform meaningful work provided us with valuable insights in how typical Office customers learn and use the Office 2007 user interface.

In this series of articles, I will share with you some of the lessons we learned from this study and how the information was used in combination with many other sources of customer research to improve Office 2007 before Beta 2 got out the door.

Now before you jump out of your chair and scream, "ONLY TWELVE USERS," let me explain a bit more about the purpose of this study.

Yes, the sample size for this specific study was small. And yes, the study only included people in one company in the Pacific Northwest. It is true that for these reasons the results of this study aren't statistically significant.

But the purpose of this study wasn't to provide quantitative results. Rather, the goal was to employ ethnographic research strategies to allow us to understand qualitatively how the new Office 2007 user interface fit into daily work lives of typical Office customers.

There are several factors which contribute to the overall Office user experience which cannot be accurately measured through satisfaction surveys or through two-hour structured studies in our usability labs.

A big part of our research strategy for the new user interface is the concept of triangulation: collecting data and feedback from many sources, with many different kinds of studies, and with widely varying sample sizes and time periods. This particular study was designed to watch how Beta 1 fared not just in the first hour or the first week, but over months and months of use.

We went to great efforts in selecting participants for this study who encompass a range of characteristics found in Office customers and who use Office applications to perform a broad range of work-related tasks. For instance, we wanted some people who mostly used Word, and others who lived in Excel or PowerPoint. We looked for people with varying proficiency with Office and people in different kinds of jobs.

This selection process, combined with the high-touch methodologies we used, gave us an incredibly valuable insight into how the user interface changes would fare in the real world--eight months before we would have usually even started to gather this kind of feedback. And because we got the feedback early, we were able to address deficiencies in the product and then measure the impact of the improvements in future betas.

Next time: Thoughts about training and migration tools from the study.