Quality Is Usability
We have a number of ongoing, long-term projects designed to help us test the
overall usability and learning curve of the new Office 12 user interface.
One of the most important tasks has been a full deployment of Office 12 Beta 1
within a local company. We chose a dozen people with all different job
types (but none of them computer or software related) and replaced their Office
2003 with Office 12 Beta 1.
On the surface, this doesn't sound that novel. Obviously, if you want to
find out how well a product works, you need to get it in front of people.
What is unusual is how early we started this study.
If you've been in an Office beta program, you know that Beta 1 is typically
extremely rough. It's designed to give people a glimpse of where the
technology is headed but not really suitable for daily work unless you're
willing to put up with a lot of bugs. We typically do not get the quality
of Beta 1 far enough along so that we would feel comfortable with "normal"
people using it. It's designed for our hard-core MVPs, partners, and
technology enthusiasts.
That said, we knew that if a long-term, longitudinal study with real people was
going to impact the product, it had to happen early enough for the feedback to
be acted in the product. By the time Beta 2 is out, we're mostly focusing
on quality, performance, and fit-and-finish issues and we don't typically
revisit the design of a major component of the product.
Thus, in early November, an unsuspecting group of volunteers at a local company
got their rock stable, menus-based Office 2003 replaced with Office 12 Beta 1.
We've learned at enormous amount from this study, and I'll be sharing many of
the results with you both anecdotally and statistically in the months ahead.
The overall feedback has been very positive and, in the places there are have
been problems, the feedback has been totally actionable. We have made many
improvements to the product to address feedback garnered from all sources about
Beta 1.
The single most interesting takeaway for me, personally, has been seeing the
incredibly strong link between the quality and the usability of the product.
Normal people simply can't tell the difference between something that's broken
and something that was designed poorly. Most of the "usability" feedback
we get is actually bug reporting.
When the quality sucks, the usability sucks. And it's not just the obvious
things like crashes and data loss; the fit-and-finish ultimately makes or breaks
the usability. When a menu disappears too quickly, or the selection
handles aren't intuitive, or when a feature doesn't work in a certain
scenario--all of these may be accounted for as bugs which will be fixed in a
later version, but these things will add up and taint your usability numbers.
One of the hardest parts of synthesizing all of the usability feedback we get is
knowing when to write off things as "probably fixed in a later build" without
neglecting real issues.
