Early in our work designing Ribbon content, we had little data to go on in terms of how different content layouts within the Ribbon would affect the usability of the features being laid out.  Being a new control, there wasn't any direct information we could fall back on to tell us how to use it most effectively.  So, we did what we almost always do in the absence of good information: use our gut feeling to make a decision and then get the prototype into the usability lab as soon as we could to start validating the design.

This was one case in which I was pretty sure I was right.  My instinct and experience told me that the chunks within a Ribbon tab should be laid out generally left-to-right in decreasing priority.  (Except of course in right-to-left languages, in which we would do the opposite.)  To me, this made perfect sense: people read left-to-right, we've watched people scan menus and toolbars for years left-to-right.  I just knew logically this would turn out to be the right decision, so we laid out the content that way in the Ribbon and headed to the lab.

(Excel "Sheet" tab in recent builds - click to view full picture)

To my surprise, some of the tabs using these left-to-right layouts were performing terribly.  We would put the key commands over to the left, and people just never found them, or they wouldn't find them until they scanned the rest of the tab first.

In short, my gut instinct and my intelligent guess were both as wrong as they could have been.

As we watched more people use the prototypes, we started to understand more the scanning process that was taking place.  Later on, we did eye-tracking studies to watch how people scanned the Ribbon and it further confirmed our theories based on the earlier tests.  It turned out that the truth wasn't a big "A-HA!" moment... it was a series of realizations of how different combinations of layouts, iconography, text labels, and position affect the usability of a layout.  In short, the truth is complicated and we needed to try to understand the relationships at work between layout, scanning pattern, and usability success.

One important thing we learned, for instance, is that many people tend to start scanning directly under the name of tab they clicked, proceed to scan around that area (both directions), and then continue to the right.  This means that we need to consider where a tab is positioned in the overall tab set as we consider the layout of that tab.  (The fact that we ship Office in 30+ localized languages makes this a non-exact science.)

Another thing we noticed was that, totally contrary to my expectation, the leftmost chunk in a tab is kind of dead space.  It's often the last thing people scan.  So we switched our thinking relative to this position and started moving commands that were consistent across the apps as much as possible into that space.  This means that you spend more time becoming peripherally aware of the same feature being in the same position and as a result you counteract the fact that you don't scan it as soon.

It's not even that straightforward.  Certain commands, such as starting a slide show, or creating a new slide, seem to have a high affinity and expectation for being on the left and do just fine in that area.  The exact kinds of layouts present in a tab (such as presence of in-ribbon galleries) also affect the usability of different relative positions, and we continue to learn more every week.

I was reading a blog entry of someone who was kind of critical and dismissive about what we're doing and our designs.  One of his criticisms was "how bad the usability of the Ribbon would be because it's got icons scattered all over of various sizes."  What we've learned is actually the opposite.  People can scan disparate patterns more easily than homogenous patterns.  When we use more toolbar-like layouts--a bunch of equally-spaced, equally-sized buttons, people scan them less quickly than when each chunk has a memorable layout.  So we actually try explicitly to vary the layouts between chunks--it helps people find the thing they're looking for more quickly.  That's something we wouldn't have known if we didn't have a commitment to watch people work.

It's becoming more clear every day that there's a nascent language at work here, and we need to keep working to become fluent in it.  Given that interaction design is just a form of communication, I guess that shouldn't be a surprise.

There's just no substitute for watching real people try to get work done in your product.  A well-designed usability methodology, along with an openness to the possibility that your initial assumptions were wrong, can help turn a design from a clunker to a winner.