Inside Deep Thought (Why the UI, Part 6)
This is the sixth part in my eight-part series of entries in which I outline some of the reasons we decided to pursue a new user interface for Office 2007.
Microsoft is tracking your every move!
Soon after you install Office 2003 on your computer, a balloon pops up asking if you would like to "Help Make Office Better." If you click on it, you are given the opportunity to enroll in something called the Microsoft Office Customer Experience Improvement Program. If you opt-in, anonymous data about how you use Office are uploaded to Microsoft occasionally in the background.
If you're the curious type, you might have wondered where your data goes. Well, today I'm here to answer the question: it goes into an Excel spreadsheet I have sitting on my desktop.
OK, back up. Back in the olden days of designing software at Microsoft (say, pre-2003), design decisions were mostly supported by guesswork. There's a classic Microsoft interview question (that I've never heard of anyone actually using) "How many gas stations are there in the United States?" Many have criticized that type of question as being feckless; personally, I agree and it's not representative of how I choose to spend my interview time with a candidate. But the rough "estimate an answer and defend it" style required to answer the gas station question was at the heart of how many design decisions used to be made at Microsoft.
Suppose you were designing the adaptive menus in Office 2000 and you wanted to know what features people use the most. Well, you start by asking a "guru" who has worked in the product for a long time. "Everyone uses AutoText a lot," the guru says. The louder the "experts" are, the more their opinions count. Then you move on to the anecdotal evidence: "I was home over Christmas, and I saw my mom using Normal View... that's probably what most beginners use." And mix in advice from the helpful expert: "most people run multi-monitor, I heard that from the guy at Best Buy."
So much of what we did was based on feel, estimation, and guesswork. How much that was true only became clear with the introduction of a technology called SQM (pronounced "skwim").
SQM, which stands for "Service Quality Monitoring" is our internal name for what because known externally as the Customer Experience Improvement Program. It works like this: Office 2003 users have the opportunity to opt-in to the program. From these people, we collect anonymous, non-traceable data points detailing how the software is used and and on what kind of hardware. (Of course, no personally identifiable data is collected whatsoever.)
As designers, we define data points we're interested in learning about and the software is instrumented to collect that data. All of the incoming data is then aggregated together on a huge server where people like me use it to help drive decisions.
Hard at work in the SQM data center.
What kind of data do we collect? We know everything from the frequency of which commands are used to the number of Outlook mail folders you have. We know which keyboard shortcuts you use. We know how much time you spend in the Calendar, and we know if you customize your toolbars. In short, we collect anything we think might be interesting and useful as long as it doesn't compromise a user's privacy.
How much data have we collected?
- About 1.3 billion sessions since we shipped Office 2003 (each session contains all the data points over a certain fixed time period.)
- Over 352 million command bar clicks in Word over the last 90 days.
Reflected in these numbers is that we don't even retain all the data points we receive... particularly, we get so much Word and Outlook data that 70% of it is thrown away.
So, one of the biggest reasons that we decided to do the new user interface for Office 12 is simply that, for the first time, we have the data we need to make intelligent decisions. Anything we would have done in the past would have been based more on guesswork and bias than on reality. Data is just one input to the design process, of course, but there's something extraordinarily empowering about knowing which commands people use often and which they don't. And knowing which commands are used in sequence with which other commands. And which commands are used 7x more with the keyboard than with the mouse. And how big people's screens are... and how much of the time they use Excel maximized... and how many documents they use at once... and which commands literally are never used... and which are used much more frequently by East Asian users... and on and on...
Knowledge is power. And having that knowledge makes this the right time to reinvent the user interface of Office.
Want to guess what is the most-used command in Microsoft Word? The top 5 commands used? Post your guesses in a comment and I'll answer in the next post.
(OK, this is a repost so a lot of you know the answer already. If you want to view the guesses from the first time I posed this question, view the comments with the original post here.)
