Chris Pratley's Office Labs and OneNote Blog : Design

One percent for art: Napkin Math a.k.a. The Calculator in OneNote 2007

Chris_Pratley — Tue, 28 Feb 2006 09:00:00 GMT

I like to love the software I use. I gather many of you do too. In the process of designing a new release of OneNote its important to keep this in mind, since it is easy to make a product that meets a lot of business criteria but has no soul.

For example, our product development process uses a fixed development budget (days, not dollars) and a (relatively) fixed ship date. We also have a high minimum bar for quality. So the only variable we are able to play with is what we do with our budget.

One way we could use the budget would be to gather the list of customer requests and simply list them in order of frequency. "Gee, a lot of people ask for drag and drop of section and page tabs, let's put that at the top". Maybe we also throw in a sense of how serious a problem each one is (its "severity"), not just how common it is (frequency). Then we just add up the cost of each until the budget is used up. You could say this is a great strategy because it is responsive to customers. But you could also say it is backwards looking, or lacking in vision, with a tendency to scatter the product all over the place and lead to bloat since we end up with essentially an unconnected laundry list of features to add.

Another way might be to look forward at where you want the product to be in 5-10 years. If you have a "roadmap", you can plan a path to achieve various milestones on it, and have a plan to get you to the type of functionality you want and thereby the business you want. For example, we want OneNote to be used by nearly everybody for "high value" activities (i.e. we are so useful they are all willing to pay us well for the product). Part of that vision for us is to be an application where it is as easy to share thoughts and project info with others as it is to use it yourself - i.e. we want to revolutionize the idea of "collaboration". This sort of long term goal tends not to show up on user request lists because they are looking at flaws in the current product. Obviously if you do this exclusively, you risk alienating your user base since the current users were attracted to your product by what it does (or almost does) today and not so much by what it might do in the future.

We naturally take a blend of both of these strategies to give enough customer love but also stay on our mission. But that isn’t quite enough. There's not necessarily a spark there which provides the soul of the product.

Consider the municipal budget for your community. It is no doubt full of pragmatic line items such as school funding, pothole filling, park maintenance, sewer upkeep, and so on. But many municipalities know that if they spent their whole budget on these things then life for their citizens would be a dull (albeit pleasant) existence. So a lot of communities have a philosophy called "1% for art". The idea is that if you simply grind down the list of priorities for the community you will run out of budget before you ever get to items such as "make our community more stimulating to live in" or "provide exciting experiences for our citizens". You probably won’t even make it out of schools and sewer maintenance. So rather than just have a long list of items ranked by universal priority, these municipalities appropriate "1%" of their budget to spend only on these whimsical things that make life interesting, such as public art or festivities. The theory is that 1% isn’t going to have a material impact on those other budget items but it achieves a wholly different goal.

So it goes with OneNote. We try to balance priorities around making existing customers happy with our longer term vision for the product (which involves having way more customers than we do now, also happy). Then we have some priorities whose future value likely exceeds their current value to us so we "over-invest" in them, such as Tablet PC support. Then we come to our version of "1% for art". With this budget category we can fund features which would not survive if they had to be directly traded off against other features in the main categories. But they serve to add soul to the product (or cool, or sex appeal, or whatever tingly word works for you). Some examples include the linked audio notes in the first release, video recording and screen clipping in 2003Sp1, audio search and OCR/search of images in OneNote 2007. And then there's a little favourite of mine in 2007 which reinforces the difference between OneNote and any other tool, as well as our focus on what people do in an app like ours: quick of-the-moment activities. That feature is Napkin Math (a.k.a. the Calculator)

When I am sitting in a meeting or just at my desk trying to work my thoughts out it often happens that I need to do a little math. In these cases I am already in OneNote, either taking notes or capturing my thoughts in some way. Now I need to know what our install size is going to be if we remove a few components, or what % of our dev budget the napkin math feature will be. So I could reach for a calculator or try to find the one built in to Windows, or maybe launch Excel. If I did, it would take awhile, it’s a context switch, and I might get "the answer" but I would have no record of the calculation or how I did it unless I typed it into OneNote. Hmm. What if I just type it in to OneNote right there in my notes? These aren't the actual numbers, but for example here's 88.7MB total install size minus three components of varying size; 4.6MB, 2.9MB, and 11.3MB, oh, and we might need to add a 6.5MB component:

88.7-4.6-2.9-11.3+6.5=

Now I just hit Space bar or Enter and I get:

88.7-4.6-2.9-11.3+6.5=76.4

Well, that was nice. What about the % of effort calculation? These aren’t real numbers either but let's say there are 11 developers who work for 4 milestones each 7 weeks long (a week is 5 days), minus 30 days each of security work, minus 7 days each of service pack work, and the total cost of the napkin math feature is 5 days, the cost of OCR is 14days, and audio search is 9 days. Let's do it:

(5+14+9)/(11*4*7*5-30*11-7*11)%= 2.47

Ok, we're at 2.5%. More than 1% but in line with what we want to spend on this category.

It's not just arithmetic. You might want some other things like trigonometry:

39*sin(30)= 19.5

Or simple functions like:

Sqrt(256)= 16

Maybe you want to force more precision. If so, just make sure one of the numbers includes the number of decimals you would like to see in the result:

4/7.0000= 0.5714

The nice thing about napkin math is that it is right there where you need it. It is just "smart paper". No need to jump to another tool, and of course your calculation is preserved for you or others to refer to later. Very handy for back of the envelope (or napkin!) estimates of market size, likely GPA, attendance at an event, budget, etc. I do want to emphasize that this is not an "equation solver" - it won’t "solve for x" or iterate or anything like that. Strictly simple arithmetic and functions. Naturally if you need something heavier duty you go to Excel or Mathematica, but this is right where you need it.

Napkin math falls into the "art" category because it clearly isn’t a top priority for our customers (zero people asked for it) or the vision for the product or any other budget category. But it is cool, it works just as I want when I need it, it is different, and it makes me love my software. We've already got a few smiles from our beta users about it.

A friend of mine told me an interesting anecdote today. One of the designers of the Windows interface prototypes (for the release *after* Vista) was showing him a cool effect where sliding a window over a background image of a bush "rustled" the leaves of the bush. My friend instinctively smiled, then found his hard headed Microsoft program manager habit came back and he almost involuntarily said "nice, but what’s the point?". The designer said, "You smiled, right? That's the point".

Your first five minutes with OneNote

Chris_Pratley — Sun, 19 Feb 2006 13:04:00 GMT

Being a new type of application has its downsides. On the one hand the field is open and we can define a whole new experience for using a PC. On the other hand, most people who see us for the first time just look at us and say "huh?" and move on. Grr.

We anticipated this problem from the start, which is why we built the little tutorial that comes with OneNote and launches when you first start OneNote 2003. (You can see it in "Help/Microsoft Office OneNote Tour" if you have 2003). That didn't really solve the problem so in SP1 we improved the original lame tutorial and added the "Helpful Tips" section, as well as got more aggressive with providing some pre-made sections and folders so people could see how and what to use the OneNote for. This seems to have helped somewhat, but there are still lot of people who just aren't willing to invest even 5 minutes to see how a new tool could help them. My Dad long maintained that "that internet thing" didn’t have any use to him, so we're going to be realistic about OneNote :-)

With 2007 we are going at this even more aggressively and also more methodically. We've been trying to narrow down exactly what people think after they start OneNote. The first step is understanding what sort of headspace they are in. They may have downloaded the trial version - in that case they already know something about the application and must have some kind of interest, but the variation is pretty great. They may have heard about it from a friend who was super-positive about OneNote, or they may have just ran into a mention of it somewhere on the net. They may have bought a computer that came with OneNote, and stumbled across it (or maybe they bought the computer because it had OneNote "for free" - we're not too worried about those people understanding our value). They might have bought one of the new versions of Office 2007 that includes OneNote and wondered what this new thing is. There are lots of ways people can end up staring at OneNote when it launches. In any case, we set ourselves a goal that within 5 minutes we should be able to get people excited enough to buy into at least trying OneNote for awhile to actually do work or school or home stuff. If we can do that we are pretty confident we can "keep them". Anything to avoid the shrug and the Close box.

To that end we've designed a new OOBE (Out Of Box Experience). When you start OneNote we'll ask you to choose a notebook type (Student, consultant, lawyer, etc) and that choice will go a long way toward helping us provide a customized experience. The notebook you choose will open with pre-set sections that should make sense to you. Students will see a note book designed to take a semester's worth of notes in various classes, do some shared projects, plus a little fun stuff. Lawyers will see something appropriate, and so on. Maybe even note flags will be customized. We'll see.

OneNote 2007 also comes with a guide notebook (since we support multiple notebooks now). The "OneNote Guide" notebook is a semi-interactive guide written in OneNote itself. That means you get to use the features of OneNote as you learn a bit about how it works and what to use it for. For example, we could talk for a long time about how you can click anywhere on a page in OneNote and type and still not convey what the usefulness of that is or the experience, or we can just have a big arrow and say, "click anywhere, for example here!" We know that many people have a low tolerance for reading explanations and want to cut away to actually try things, so the whole guide is oriented towards trying things. Note flag summaries are much more interesting if you have already got a bunch of note flags on a page for example. The guide also can be tuned for different "audiences". So if you are a student, there is a "for students" section that explains how OneNote can be useful to you as a student. It shows different features than it would show to a lawyer, for example. Not just features either - it talks about situations where OneNote can help.

We’ve been running some "focus groups" recently to fine tune the guide. Each time we have 6-10 regular ol' people come in and sit in a room where we tell them that they’ve just discovered this new application and ask them to start OneNote. Then we just sit back and wait.

With the first focus group we succeeded at our first major goal, which was to explain what OneNote is. We found that without the guide a lot of people couldn’t see the difference between OneNote and a word processor, since the blank page and the fact that you can type and make text bold seemed too familiar. So the guide first tries to introduce the concept of a notebook with organization. It introduces notebooks, sections, and pages, and that went pretty well. We didn’t do too well with our next two goals though, which were to show people what OneNote could do and why it mattered to them.

We made some modifications to the guide to give example scenarios of how you would use OneNote. We also torqued the language around to talk about problems the user might be having and how to use OneNote to solve them, not just what OneNote can do. We took advantage of new features in OneNote to design the guide while simultaneously showing people how useful it can be. For example we have a "Table of Contents" page where lists of links are organized spatially on the page in clusters related to task or activities. We chose this layout specifically to drive home that this isn’t a word processor with a linear organization on the page. These links point off to other pages in the Guide notebook so people learn that a OneNote notebook is sort of like a mini website you can easily make to manage your information. We added a "more cool features" section so people could dive off and gravitate toward what interested them since we had a theory that each person adopts a product not because of the aggregate of its functionality but mainly because it does one or two things they really like, so by offering a smörgåsbord we could get a "bite" on one or more features and they would be sold.

Our second set of focus groups went much better. We asked people to rate the value of the product and their likelihood of using it further and the scores went way up (with a lot of people giving us 5 out of 5 on the ratings). We're still working the whole OOBE though, since it isn't firing on all cylinders for all people yet. Feedback from beta 2 will help a lot.

I would love for you all to share your impressions of OneNote in the first five minutes (if you remember). Also please share what friends or colleagues said about their five minutes, especially if they did not choose to start using OneNote right away.

Clippy and User Experiences

Chris_Pratley — Thu, 06 May 2004 03:30:00 GMT

Some people have asked me to write about user experience design (as in the whole shebang, not just interface elements or features), and also about the Office Assistant (sometimes known by the default assistant, "Clippit" often referred to as "Clippy", or other ahem, "nicknames"). Since they are somewhat related, I'll cover both here. Also, I'll use the term "user experience" (UX) to describe the whole user interaction with the software/PC, and the term "user interface" (UI) to discuss the actual elements of the interface.

Although many people associate the Assistant with Word, it is more accurate to say it was a feature of Office, since it was implemented by our "shared" team that builds features used across Office, and it appeared in several apps at once in 1997. As such, I only know about it second hand - I talked a lot with the people who worked on it, but I do not have the authority on details. Maybe someone will blog about that. All this to say I will be giving you my perspective, not the "horse's mouth" version, so I may get some details wrong.

If you look at the interface of Office applications, you might think they have not changed all that much over the last ten years - and I think I’d agree with that at the highest level. There are still menus and dialogs after all. However, that doesn't mean we don't consider radical approaches on a regular basis. And some major new elements have shown up over the years that have fit right in, to the point where people forget they are new to Office (e.g. Task Panes, which debuted in Office XP). Many of our biggest customers actually ask us not to change the UI in any significant way, since they want to avoid retraining users after an upgrade.

As an example, when the web first appeared, we looked at that interface of links and pages and, like many, appreciated the ease with which you could navigate through seemingly infinite amounts of information. At the time of course, everything web was widely viewed as "the future" and the "only way to go" by pundits and even many inside Microsoft. This was not true in Office particularly though, since we tend to be a little more considered when it comes to changing something as important as the Office UX. (N.B. we did fall under the web spell in the area of file format though - that's why you see the broad HTML support in Office - more than would probably make sense if the future could have been seen.)

One of the many prototype UXs we looked at in 1994-96 (for Office97 or 2000) was a web-style interface for Office applications. This set of prototypes tried to group functionality in different ways on "pages". From any page, there were links to other pages that had possibly related commands. For example, maybe on one page you could change the font, and on that page there were also links to related things to text or to fonts, like text shading, strikethrough, insert hyperlink, etc. This approach didn't test too well in usability though, since as we had surmised, a web style interface is useful for browsing, not directed usage. If you know where you are going, you want to get there as quickly as possible, and always by the same path. With the web in 1995, usually you were surfing when following links, not looking for a specific thing. When you wanted something specific, you use favorites (i.e. a menu) (remember that search engines that actually could get you where you wanted came later). Menus and hierarchy are great at directed use. We liked the "see also" idea though, and this showed up in a limited way in the task panes eventually. There was also some enthusiasm for a kind of "command search" that like a search engine would let you find commands by entering keywords. This became part of the help system. The web interface itself however never saw the light of day.

The Office Assistant came about as an idea from Microsoft Research around social interfaces. There was a lot of research at the time around how many people interact with computers by anthropomorphizing them - that is, treating them like a person rather than as a tool. I guess the theory was that if you could provide an interface for the computer that expressed emotion and that you could interact with, you would be less likely to develop animosity toward your PC (much like the impassive camera lens of HAL9000 caused tension), and would actually be encouraged to learn and interact.

As is often the case, the transition from research to reality caused some compromises in the design, and the result was the "Agent" which was a system service that could be installed on Windows and used by any application. The Agent provided the animated character Office called the "Assistant". You could install additional agents if you wanted - Office97 shipped with seven, although most people stuck with the default, Clippit a.k.a. Clippy (the default was different in other countries - e.g. in Japan it was Kaeru the Dolphin). Personally I would have selected Rocky the puppy as default, since he was cute, and animated to be subservient and harmless, whereas Clippy was sassy and annoying. But there were reasons to not choose Rocky - cultural, focus group preference - some fuzzy thing I can't remember.

Now the Agent interface was more limited than the original researchers had specified and tested. And on top of that, the kind of information the researchers expected the Agent to provide, such as "I'm busy searching for that, just a moment", or "I'm ready to help you" was hard to convey. There were many reasons for that - sometimes office code simply didn’t allow for good notifications about tasks the applications were doing while they were doing them, and the reasons for some slowdowns were not known or were not predictable (e.g. network traffic, memory swapping, unexpected disk activity, etc).

There were also differences of opinion. For example, the original idea was to show the Assistant moving about when the application was ready for user input, since this seemed to indicate that the application was not "hung", and seemed to project a "ready to help" atmosphere. But some people in Office felt that the animations were too distracting. After all, often you were just sitting thinking in front of your PC, not just walking up to it after a break - and the animations made it hard for some people to think. So these were toned down significantly, to where the Assistant in "wait" mode barely moves at all.

A big part of the Assistant plan was to use it as the gateway to help. You could click on the Assistant and ask it questions in normal English (or other language depending on your version of Office). This was supposed to reinforce the relationship with the user and encourage them to ask more questions. (it did not do this as it turned out - people still overwhelmingly type a single word in the hopes that will get them the answer)

The most famous use of the Assistant was to deliver "tips". The idea was that the software could monitor your actions, and if you did the same "dumb" thing all the time, we should be able to show you a much faster way to do it. Everybody says this to us at some point: "I am sure there is a better way, but I just don’t have time to figure it out". The hope was that the Assistant could introduce you to powerful new capabilities, or remove drudgery, based on monitoring what you did.

Although there were tips all over Office, the most famous "tip" was the "It looks like you’re writing a letter" tip in Word. Word97 had just introduced a new letter wizard that was designed to make writing a simple document like a letter much easier. We saw so many users have trouble with this simple task, the theory was that with the wizard you would have no need to learn about right-align tabs, tab stops, etc to get address blocks right - just use the wizard.

But the problem was that people did not discover the wizard - they would just keep trying their broken ways of trying to get a letter to come out right. So this seemed like a perfect instance where the Assistant could help. If we could only find a trigger to figure out when the user was probably writing a letter…

The string "Dear <blank>", where blank was a set of words, was chosen. Of course, this tested well in usability, but this was a great example of an effect I have described before (See here), where the design of the feature was optimized for the first use based on usability testing, and not for continued usage. Compounding things was that this tip did not have a way for the user to turn it off, and it was a little too persistent before giving up.

We gave one more try to make the assistant better by using the new agent technology in 2000 (the assistant no longer needed its own window, which blocked the content underneath), and some rough edges were taken off the tips and the way they appeared (tips had multiple settings for "aggressiveness" - most tips were taken down a notch or two on the five notch scale), and we made it easier to turn off the assistant permanently (although a bug made this fix not work for some people). Although this made things better, the fundamental approach was not really working well enough.

It may be surprising to many who read this blog, but the Assistant was actually a wash in terms of user acceptance. Many users told us that they really liked it and found it useful, something which technical people have a hard time believing, since they were the ones who pretty much uniformly didn't like the assistant. In terms of population, the numbers were split about 50/50 for/against its value.

But only 50% is not good enough when a significant chunk of the other 50% felt strongly negative toward the Assistant. So eventually (Office XP), the Assistant was turned off by default and the issues that made it come back sometimes were finally exorcised. The marketing team even made light of this in a little campaign about Clippy being retired that some may remember.

So, was the Assistant a success, failure or something in between? If so, why? If you think the Assistant idea was bad, why exactly? It is interesting to learn from these experiences to try to move the state of the art forward. I have heard that the researchers who originally supported the idea claimed that the idea was sound, but the implementation in Office was inadequate and flawed. Many in Office would say that the idea would not have worked acceptably well even with an ideal implementation. Still others would say it did pretty well, and with a little more work could have been made useful for those who would use it, and not annoying for others.

Designs, anyone?

Chris_Pratley — Thu, 15 Apr 2004 08:53:00 GMT

Excellent. Lots of feedback. But almost all from developers it seems. Having an API for OneNote is certainly the plan, so it is good to hear how people hope it would be used. No need to convince me that it is necessary. And I agree with all the sentiments about bloat, etc.

The add-in argument I think is missing some key factors: add-ins done for fun are not really industrial strength code, or UI design, and are not available in different languages. They haven't been security reviewed. The developer who built it might move on to something else since they are just doing it as a hobby anyway.

Remember that for people who can't code, an API is not providing them value directly. If you tell a customer that they bought a product, but they need to write an add-in to get the functionality they want, or track down on the net something someone else wrote in their spare time, that isn't too customer-friendly. For the great majority of people the feature does not really exist and never will. The Office family of products go to a large audience and not just the sophisticated users you find here who would write or search out and download code. The ordinary person wants things packaged up nicely, debugged, in their language and so on. Of course, you could argue that only the sophisticated users would blog, so no problem. But then there are people who say that blogs are taking off and becoming mainstream - there are millions of them already after all...

FWIW there is a rich API for Word2003, and rich XML support, and even some blogging add-ins, but most people aren’t aware of them. Even with hundreds of custom-use add-ins available for different uses the total number of users do not add up to a great majority of users taking advantage of them. It is still a small minority really even taken collectively, given the huge numbers of units involved with Word.

I'm curious about the lack of interest in blogging support from the people who commented. Is this because there is a belief it wouldn't age well, given that blogs are evolving quickly? Or is it because there's a sense that there are already too many blogging tools available? Or is it that the app is not suited for this? Where's Scoble when you need him?

Today you can use OneNote for blogging if you have a blog site that supports receiving blog entries via email. Greg Hughes is doing that here.

OneNote has an edge over other tools for blogging in that you can post multimedia entries, including handwriting, ink-annotated photos, audio, etc. It is also a natural place to manage a lot of different entries you are working on, especially if like me you tend to write on different topics and take some time on each entry and don't simply post whatever floats across your brain at the moment (not that there's anything wrong with that :-))

I'm surprised that the people commenting were not offering any designs - even of a potential add-in. To me it is too simplistic to say that selecting some text and posting it is what is meant by blogging support in OneNote. Offline use is interesting I think. Also being able to view all my past posts while offline is useful. What about blogging internal to a corporation, where their might be a little more interactive or collaborative use of blogs?

OneNote is used by many people as a research tool even more than as a note-taking tool. People grab stuff off the web and paste it into OneNote to gather info on a topic. Handling RSS feeds seems a natural to me to extend that (as Brian noted in the feedback). It is a nice complement to manual information gathering.

So, any thoughts on what the design of blogging or RSS support would be? Even if you would do it via an add-in, how would it work? Blow me away, please...

Design a blogging feature for OneNote

Chris_Pratley — Wed, 14 Apr 2004 08:08:00 GMT

I thought it might be fun to try a little interactive design experience. There's been a lot of discussion about blogging from OneNote, and how it would make a good tool for managing posts, etc. So it is easy to say "hey, add support for blogging to OneNote!". But if you've read my previous posts on how software feature decisions get made (link), then you know it is not as simple as just wanting something. The feature has to be justified on several levels: value to customer, development/test cost, completeness of solution, appropriateness for the application, likelihood of being competitive with alternatives, etc.

So I thought it might be a fun exercise to take y'all through this process for something that I am sure most of you can relate to: reading and consuming blogs. Now, I want to be clear that as we do this, I am not foreshadowing or promising anything about future versions of OneNote. This is only an exercise - however, I am going to apply the same rigour we would use for any proposed feature for OneNote.

First a word of caution - don’t let your enthusiasm for blogging carry you away. We're a business, so we don't do stuff because "its cool", or "I'd love it" or "blogging and RSS *rules* so all apps must support it". Or "Blogging is the answer. Now what was the question?" You have to make a business justification for the feature and your approach.

So, let's dive in. How should blogging from OneNote work? Is that even the right first step? Maybe we should instead subscribe to RSS feeds and consume blogs rather than produce them? Why are we even considering this? Where's the value in any of this vs. single-purpose free software tools? Is it in integration with the rest of your info in OneNote?

Is the power of OneNote that you can "blog offline" then sync up later? If so, is this automatic or a manual "publish now" button that only works when online? Or is it just a pretty good text editor (not as good as Word though, so why use OneNote when Word is there?)

What's the scenario? Are you writing single entries in OneNote and publishing them, then forgetting them (like Word docs)? Or are you keeping a OneNote section that matches your blog one page per entry, so at any time you can change or update past entries and upload the changes in bulk? If we are consuming RSS, do we have a feed going directly into a OneNote section? If so, does that scale to hundreds or thousands of items? Are we blogging for fun, or for business? If the latter we can hope to generate much more revenue than for pleasure, but it is harder to show that blogging is viable in an organization. What's the business case you would make to a customer if we did this for business users? Do we need to integrate with portal software, etc...

Well, that's enough questions. I'm curious to hear what you think. I've created a new category for this discussion, so we can continue it while the main part of my blog meanders on...

Awards for OneNote

Chris_Pratley — Wed, 14 Apr 2004 07:33:00 GMT

We've had a good week here last week on the OneNote team. First, we heard that we had won a Microsoft internal award for “Best Software User Experience” at our Design Day 2004. This is an event that is held each year as a kind of internal conference on design. One of our users had entered us into the competition, and we ended up winning in our category. It is nice to be recognized by your peers...

Then later that afternoon, we heard that eWeek had awarded us its “eWeek eXcellence” award in the personal productivity category. You can read more about it here: http://www.eweek.com/article2/0,1759,1559948,00.asp.

We had previously won an award in Italy and in the UK (PC Plus Editor's choice). So that brings our “medal count” to four.

I usually discount awards since it is so easy for me to see the flaws in anything I work on, but today I was holding the actual lucite block for the Design Day 2004 award and that made it a little more real. It was kind of weird to feel that other people had decided something you and your friends had made was worth giving you an *award* for.

Hmm. Must...win...more...awards...

Avoiding typical pitfalls

Chris_Pratley — Fri, 27 Feb 2004 10:10:00 GMT

So, how to make sure we didn't have a typical "v.1" product with OneNote? As I mentioned last post, it isn't as simple working on the product until it is "perfect" in design - that approach doesn’t work. Takes too long and you don’t really understand what "perfect" is until after you ship it (if you ever do).

I also knew that shipping the equivalent of "version 3" as the first version wasn't really possible. If we quantify the amount of development put into a typical release as "X", then a version 3 has something like 3X worth of work put into it. But depending on how far off you were with version 1, that first X might only count for 1/2, and the second X is only 3/4, so in the end due to inefficiencies and rework caused by inaccurate initial design, you end up with something more like 2X-2.5X worth of "features" in version 3 depending on how far off you were with the first idea, and how fast you were at correcting. So shipping a version 1 that was really version 3 wouldn't be possible - we simply couldn't put 2X of dev effort into the first version. Instead my goal was to be as "on track" as version 3 in terms of design with our 1X dev effort. That is, instead of half of X, we'd get as close to a full X worth of useful stuff the first go round. It would feel like version 3, but be much leaner.

Most new software projects seem to be developed by technologists who get excited about a technical solution, then go search for someone who hopefully has a problem this solution solves (the hammer looking for a nail). The problem with this approach is that very often you are solving a problem hardly anyone cares about, or at least does not consider serious compared to the problems they really care about and need to spend their money on. In fact this detail is the bane of most startups. Customers have a fixed pot of cash. If you ask them what they are going to spend money on, they're going to spend it on their top priorities in order, until they run out. Technologists get confused by their "solution" looking for a problem - they think that solving a problem means people will buy their product. But even if you are solving a common problem, if that problem is not high enough on a customer's priority list, there won't be any money to spend on it.

With OneNote, as I described back in OneNote Genesis, rather than being a hammer looking for a nail, I just looked to see what problems people were having without thinking about what technology we might develop. So right there was a better start than most products get.

The next thing that destroys new products is lack of focus on solving a (possibly small) set of problems completely rather than taking a whack at a many more than that (See OneNote and version 1). For OneNote, we built 5 user scenarios that matched the problems we had seen, and determined that the product would meet those scenarios, and do them well, and everything else would wait until next time.

The next tar pit is in designing the features to solve problems in those user scenarios. A typical disaster product solves them the way the designers would personally solve them. But the designers are computer nerds, not normal people. There is a *lot* of software out there that was clearly designed to be useful to the creator of the software, and if anyone else liked it too, bonus! To get around this a good designer collects data and meets with lots of real people, ideally in their natural habitat (cf. Contextual Inquiry and other approaches). You can also do surveys, focus groups, interviews, etc - there are lots of ways to get quantitative and qualitative data.

Ok, now you're swimming in data - but be careful not to drown. The next pitfall is in misusing data, or believing data and not your own common sense. A skilled design team maintains a vision for the product that is "informed" by data, not formed by the data. I have seen numerous projects that were led astray by some factoid or other that the design team took as gospel and lost their way with. Not to be too hard on them, but Bob was like that (usability data can be disastrous without a strong team to interpret it). Office itself has many features that appeared because data took over. e.g. in Word you can find Text Effects (Format/Font/Text Effects tab). This was added back when the web seemed to be the future (1995), and studies showed Word "had to remain relevant" by adding dynamic content to its documents for on line consumption. Wow…

So if you have made it this far, you can write up some specs that represent features designed to support real scenarios that solve problems that matter to potential customers and you have data to show it. Now you turn the dev team loose, and they build something. What did they build? If you assume they built what you designed, you're in trouble - no one writes specs that well. And if you assume that your designs will actually work as you hope even if implemented as you desired, you're really in trouble. So working closely with dev and test to make sure that what is taking shape is what you intended, and then trying out what is there, seeing where it fails, and adjusting the design before you go too far in the wrong direction is key. Too often a product team simply ships what they designed originally, or whatever was built without managing it to turn out right.

OneNote did something different from Office at this point. As soon as we could get the product to stay up and running, we put it in front of a set of real people and asked them to use it. See Field Trial for a description of this. Then, since we still had the flexibility because we had done this trial so early, we made significant design changes to OneNote in time for its first beta - something a typical Office product can't do since it doesn't have the feedback. Then we took a gamble and made significant changes after the beta – something Office never does since it needs to ship at known high quality. OneNote could do this because we decided it was better to ship a correctly designed feature set in our v.1 than it was to lock down for stability like Office does, since Office has decades of valuable features already that people depend on which could not be risked. We didn't have that risk, since we had only a few features and they were all near the surface where they get used a lot. This meant we could rely on Watson telling us if we had added any hidden nasty crashes and hangs, so we weren't totally in the dark. There was a "Technical Refresh" after beta, which allowed us to collect Watson data before shipping, so we did our stabilization by reading off and fixing the top problems reported from “the wild”.

For the next release, since we actually have a functioning product now, we expect to use this technique even earlier during the development of version 2 (or 12, in Office versioning). We expect to have people outside the Office team who are internal users and possibly some limited external people use (a.k.a. “Dogfood”) our first "milestone" build - when we are less than a third done with coding. And we'll do that throughout the project, since we don't have to wait until “alpha“ now. And since we'll be a little more mature as a product that has real customers who rely on us, we'll have to be a little less reckless near the end.

BTW, thanks everyone for your support to continue blogging.

Schools of development

Chris_Pratley — Wed, 25 Feb 2004 07:35:00 GMT

I was thinking to write about how software is made. First, I'll start by describing three methods of developing software. I am sure there are more, and someone has probably written books on this, but hey, this is an informal medium. Here they are:

The "wait until its perfect" school. This approach believes that software needs to be "done" before it is released to the public. Done is defined as having all the conceivable useful features, polished to a gleam, and of course "bug-free" (see my previous posts on that fallacy). These products tend to take a really long time to come to market, generally miss their opportunity since someone with a slightly lower standard than "done" has beaten them to the punch and worst of all, they spent so much effort on getting things perfect they forgot to actually try the product out on real people and make sure they weren't missing the boat and adding things no one cared about. Several now-distant competitors to Office products followed this school (one of my favorites was the release of a Japanese word processor that touted its new version was based entirely on a component architecture! Talk about ivory tower and completely disconnected from customers, who couldn't care less about the architecture - not only that it was slow and a memory hog, like most componentized software)
The "we'll just release a new build" school. This version develops software that sort of works, then sends it out for feedback as version 0.7.1.0145 or whatever. They get some feedback, make changes and make a new build with a slightly higher build number, like 0.7.1.0146. In fact, they make a change and produce an update whenever they hear about a problem. This software usually never actually ships - it just gets slowly better although often only in increments. In fact it may never make a great leap in innovation, since it is constantly in a state of trying to get closer to a specific goal. The response to any problem is simply "we'll just release a new build". This approach is fine for hobbyists or low-usage scenarios, where the customer or user doesn’t mind updating their software all the time, or redeploying to their few machines (e.g. servers). It doesn’t work so well for mass-market software due to the cost of deploying the new build being prohibitive for many customers, and because the channels for getting updates to "normal" people are poor. Also end users expect a product to at least have the appearance of "done" - no one buys a TV that is still under development.
The "ship early, ship often" school. This is the one that most client software that Microsoft makes has followed. The theory goes that if you try to plan too much before you ship your first product (wait until its “perfect”), you will not be able build a truly useful product since you don't really understand who your future users are yet and what they will find appealing in the product. So the best thing is to get something out there, understand what is appealing and what isn't from the “early adopter” feedback, then ship another version that responds to that feedback as soon as you can. Typically, version 2 starts before the feedback from version 1 comes in, so version 2 is usually a polish of the partially misguided version 1, and version 3 is the real re-work to make the product what its prospective customer base really wants. This is where the "Microsoft doesn’t get it right until version 3" axiom comes from. One strength of this school is that it generally beats out the "wait until its perfect" school in the Darwinian world of the free market, since it gets access to real world feedback and goes through more generations than the "perfection" approach does, resulting in a product optimized for real customers sooner.

For OneNote, I wanted to do things differently. It seems obvious, but I wondered why we couldn't ship "version 3" the first time around. Why ship the product before getting substantial feedback on whether it was the right set of features - in fact even a worthwhile product or not? The standard answer is that to do so would be following the “wait until its perfect” approach: you couldn't get substantial feedback until version 1 was more or less design complete. If the user feedback showed massive changes were needed, you'd essentially be skipping version 1 and going on to version 2, taking way too long to get something out for the real world usage that you really need. I agreed, but I felt that the problem that forced the skip to version 2 was that the standard Office way of developing software did not bring in user feedback soon enough. In fact usually broad user feedback is accepted only late in the process, to determine if the software is stable and works well in the huge variety of real-world configurations. The design has been locked down early on. Of course usability tests are done on the designs and they are adjusted to make them work in the lab, but getting deep feedback from real people in real situations is hard for Office - even sending out a beta to hundreds of thousands of people only generates feedback from a few thousand. There are reasons for that - a lot of it is just that people assume we know what we are doing and don't question the design of features even when they don’t work for them. I could go on about the difficulty of getting beta feedback another time. But for OneNote, we decided to try to do things differently, which I'll talk about in my next post.

OneNote and Version 1

Chris_Pratley — Wed, 11 Feb 2004 07:07:00 GMT

Let's say you manage a development team that has a few developers, a few testers, and a handful of Program Managers. You've already laid out a "vision" for a product you plan to build, and most everyone privy to the plan seems to think the idea is solid enough to be valuable and maybe even successful. You’ve got a stable environment and funding for the time being. You start laying out the development plans based on the rough "specs" you have laid out. Of course it is still early so the specs lack detail, but you can at least estimate the work at the level of developer-month (what one developer can do in a month, including bug fixing). So you do some rough calculations and determine that it will take about 8 years of solid development to deliver on your “vision”, taking into account stability work, bug fixing, and of course a whole lot of features. Of course, long before you figure out that it takes “exactly” 8 years you stop bothering and start thinking about what to do in the shorter term.

Ok, so producing a new product at that timescale is clearly not practical. Your funding will dry up, or the market will move on and leave your product obsolete long before you ship it. Maybe you could put more people on it. But then you run into one of the rules of thumb of software, which is that more people make things go slower, not faster. Yes, more code is written quickly, but that is actually the problem. (it is sort of like asking 6 people to write a novel in 2 months rather than having a single author do it in a year - not particularly effective)

You determine you really need to get into the market in about 2-3 years. You now have some options. You could build a foundation for the 8-year product vision - a skeleton framework that is not fleshed out. That would be like shipping what you have after 2 years of the 8 year project. But customers rarely find houses with foundation and framing but no walls to be worth paying for, even if it is the frame for a mansion.

Another option would be to build a room at a time. Of course you have a plan for the whole house, but for starters, you'll build a complete dwelling which is just the entranceway and the living room. You're careful to finish it out nicely so it is livable, and even throw in a fireplace along with a nice TV and a sofa. Now you have a nice little chalet which people will pay money for. Some will even say it is just right. But most will say "I love it, but I want more - a kitchen, bathroom and bedroom would be great." Someone else says they also want a garage and a play room. Someone else says they'd like an office. But importantly, many people are pleased with the chalet, and recommend it to their friends, even though they often add that they'd really like it to be more of a condo or a mansion. After the chalet is open for business, you get to work on the next extension - a kitchen and bathroom, as well as raising the ceiling of the living room to make it more elegant. Once you’re done with that, more people will be happy but there will still be quite a few saying you're missing a room. Eventually you'll get the mansion built, but it will take you longer than it would have taken to build it straight through. More importantly, you'll have built the mansion using feedback from the people using the chalet and other stages, so you can make sure it is designed for exactly what they need, not what you imagined they would need years ago.

Naturally, this is about OneNote and its development process. Sometimes people tell me it is a great app and they love it. Others say that too, but they have a list of additions they would like. Still others say it is unbelievable that we would release the product without X feature they need, which to them is the whole point, and what we delivered is useless.

The challenge for the team making the version 1 application is to build that chalet - the subset of the entire plan which has value to enough people all by itself. Where you see most version 1 products go wrong is that they build the chalet but they forget a wall, or they forget windows and instead build a hallway leading nowhere.

There are lots of reasons for this - that will be the subject of a future post.

Why Doesn't OneNote have Feature X?

Chris_Pratley — Sat, 31 Jan 2004 21:19:00 GMT

Who decides what features go into a software product? If a software product doesn't have a feature, why is that?

My entire career at Microsoft has been as a Program Manager. You can probably read elsewhere exactly what that means, but for now we'll just say that the role is one of "make-it-happen" guy. Another way to say that is if anything is not successful during a project, the program manager shares the blame since they were supposed to foresee it and do something to make sure it didn't go wrong. Program managers do a wide variety of things - they speak to customers or potential customers, they read about technology, they try to understand businesses and individuals and what their productivity problems might be. The higher level program managers also hammer out a "vision" for a product. This is often a collaborative effort, involving many other program managers in other products. The idea is to figure out the optimal set of things we can build to make our products valuable to customers, which in turn leads to revenue, etc.

During the course of a software project, we have many processes designed to cull poor ideas out of the set we have thought of, or to remove ideas that might be great but do not fit the grander themes we have chosen for the release. In Office years ago we used to just do a lot of things that seemed valuable, but we found that when you go to explain what the product is, a thousand disconnected things are much harder to articulate than a thousand things that support three big value points.

At some point when we think we have a pretty good idea of the most important things we are sure we are going to do, the program managers start figuring out what these will look like and how they will work. As part of this process they build design documents. These are increasingly visual click-throughs (sometimes just handwritten drawings) that give us a sense of how a feature (one with User Interface) will work. This is an organic process of design, which is both highly individual at times, and also highly collaborative.

Some people imagine that BillG simply orders us to do what he thinks is important. If that were true, none of us would work at Microsoft. The great thing about the company is that you have the flexibility to do what you think is right, and make decisions at your level. Of course, you don’t make decisions in a vacuum - you spend a lot of time convincing your peers that your view is correct. This process helps you hone your idea to make it better based on feedback, or convinces you and others that it is not such a great idea.

In order to move things forward, we have a set of meetings called "adds/cuts". As we build up the list of feature ideas, it inevitably grows way beyond what we could implement in a given two year time frame (the normal release cycle for Office products). For example, with OneNote, we had so many ideas that we had to ruthlessly cut out anything that wasn't strictly required for the product to function, and then carefully used the small remaining budget to enable 10-15 features that we thought were central to the "version 1" goals we had: make a useful personal information capture and retrieval tool. Having a clear goal is critical to making a good software product. We used the vision document I wrote in early 2001 as a guide and made sure that we were all in sync on what we were actually building. It had to support about five key scenarios: note taking (both externally and self driven), researching, organizing, and retrieval.

At times people would become enamoured of some idea or another in isolation, but the shared vision, the requirements to make the key scenarios work, and the very limited budget for non-necessary features meant we had to drop anything that was off-mission. "Save it for version 2".

Now, I often read or hear from users of OneNote (or Word, or whatever) that we don't have Feature X, and that must be because we didn’t think of it. I smile at this, because it is sort of cute to think that we just sit around dreaming stuff up, and whatever we dream up goes into the product, as if development realities didn’t exist and we are limited only by our imagination. Or, I suppose I could view it as insulting - that what is in the product is by definition all we could think of. The reality is that we had enough ideas to fill the features lists of at least the next three versions, and the only reason we didn’t have more was because we were forcing ourselves not to keep dreaming up new stuff and to get cracking on defining the details of what we had decided to do.

So, why don't we have an extensibility API? Of course it MUST be that we didn't think of it, right? Or that we didn't realize that it was important. Say the same for drag and drop arrangement of sections and folders and pages, or multiple views on a notebook, or real-time note sharing, or PocketPC integration, or rich integration with the Journal application on TabletPC, etc. etc.

If we had allowed ourselves to go off and do these things, other features currently in the product would not exist, and people would be saying: "What were they thinking? This product isn’t even useful - why did they build an extensibility model for it instead of putting in the features that make it worth extending?". Of course, we don't always make the right call - sometimes our judgment slips and features get done because we thought they were cool, rather than because they were critical. Or the process gets us – we don’t discover our design was incomplete until too late, and there is not enough time to fix it properly. We don’t think of everything of course - but with more than two years spent every day thinking about the problem space, odds are we have thought about it and have decided we can’t afford to do it (this time around). Keep your suggestions coming through – even if the idea isn’t new to us, every time we hear an idea it is a vote for doing that feature in the future.

The Acid Usability Test

Chris_Pratley — Sat, 31 Jan 2004 10:28:00 GMT

Designing software is fun fun fun. During the OneNote project we had countless hours of brainstorming design sessions full of laughs and great ideas. There's really nothing quite as intellectually stimulating as a tough intellectual problem, 3-5 smart, articulate people, and a whiteboard. Ideas come and go. Analogies drawn. Metaphors extended. One of my goals in such meetings to is to inject hilarity where possible, and some of the deepest laughs I have ever had have come from those sessions. Surfing the "froth" at the tip of the idea wavefront in sessions like these is incomparable.

Then, the test comes. The idea we have laughed and cried over and finally reached an agreement on is prototyped, and we put it in front of "normal people". These heathens then have the gall to not understand how to use the feature we have designed, or not discover it among all the other stuff, or otherwise demonstrate through their confounded normality that our ivory-tower design sessions have produced a masterpiece of hokum. The program manager "owner" of the design drops his or her tail, and sulks through the remaining sessions (usually 6-8 in total) glumly verifying that yes, seven normal people out of eight could not make heads or tails of their masterwork.

By the next morning, the feedback has been internalized, the design ideas flow again, and we try once more to come up with the design which somehow meets all the criteria, fits within all the constraints, actually solves the problem at hand, and this time will in fact be usable by the dreaded "user".

Iterating in this way, we can work out most of the fundamental errors in a product's design. But usability tests are not a panacea. In fact they can be quite dangerous. Although they ostensibly test normal people, these kind souls are people who come to the hallowed halls of Microsoft in Redmond. They are very aware that they are going to be using something new, and they should be on the lookout for changes - plus they really want to be helpful, so they take extra special care to examine what’s in front of them. This is all quite unlike true normal people, who don’t have time to notice software and just need to get their work done.

Another bias that usability tests introduce is the "optimization for first use" phenomenon. You've probably seen this. There's a feature you use which seems innocuous the first time you use it, but as you use it more and more, it bugs you that it takes so many steps to get it done. Or that it seems so verbose, with a lot of user interface for something which is really quite simple. What has happened is that the usability test showed that people who have never seen a feature before have trouble with it in the first hour of using it. So the designer makes the feature hold your hand through the process. That improves the results in the test, but ruins the feature for people who know what they are doing.

Remember Microsoft Bob? That was an extreme. People who didn't know anything about computers could use Bob to write a letter. But Bob was so optimized for the first time user that even those utter novices would quickly tire of its helpfulness and want to move on to something that got out of their way and just let them do their thing.

In or Out? That is the question.

Chris_Pratley — Sat, 31 Jan 2004 07:46:00 GMT

Some people ask me why OneNote is a standalone application, and not part of one of the Office suites (i.e. not "in the box").

There are advantages to being in the box for sure. One is distribution. A lot of people will see a new product if it is included in a bundle with Word, Excel, Outlook and PowerPoint. And maybe some people will be swayed to buy a new version of Office if it includes a new application they are interested in.

On the other hand, it is hard to build a new business. For one, what is the price of a new version of Office that includes a new application? Well, it is probably the same as it always was - people are not keen on price increases, even when they get new applications in the box. That means there isn't an increase in revenue, except an incremental one if more copies of the new Office are sold. And there is another issue - you can’t put things that have value in their own right into Office "for free". Instead, you have to tweak the effective discount on the other products in the box (e.g. Word) to take into account the fact that you put another $X of value in the box but are still charging the same overall price. Then some rules around how much discounting you are allowed to do start to kick in. That means you might have to lower the price of the standalone versions of some of these products. Ok, so at some point we're losing track of the goal here, which is to increase overall revenue.

Being a standalone application is more fun. You can measure your success (or lack of it) directly. You are also free from all this discounting price waterfall mumbo-jumbo. You can set your price and discount the way you want to. But you have some huge barriers. No one knows you exist. And you have no advertising budget. But theoretically, if you ever do succeed in becoming a mainstream must-have application, then you can become a significant business on your own. And that will be in addition to Office, not replacing part of it. That's the theory anyway.

When we announced that we were going to be standalone only and not part of Office, some of the people in the press who track our every move thought that meant we weren't "serious" about OneNote. What they didn’t realize was that in fact we are very serious - that's why we are a standalone application. Naturally this explanation didn’t faze them (see "the myth" below). One of the things about being Microsoft is that people don't believe you - no matter how much sense you make. I am tempted sometimes to try reverse psychology to get my point across, but haven’t yet had the cojones to do that in any official capacity. Then again if I did of course I couldn't tell you :-)

Humility and handwriting

Chris_Pratley — Sat, 31 Jan 2004 04:23:00 GMT

Designing the actual experience of handwriting in OneNote 2003 was quite a challenge. We didn’t want to just replicate paper, since that didn’t seem to be adding enough value. So we got quite interested in the idea of trying to determine the structure of ink. That is very hard and the subject of ongoing research, so we decided to experiment with a different approach. We’d of course let you write anywhere, but we would also show you special places to write (called “affordances” in the jargon) where we would interpret your ink as part of an outline or list structure. If any of you remember the first public beta of OneNote and had a TabletPC to try it on, there were “writing guides” visible on the page to show you where you could write to tell OneNote to link two items together, or to continue a line. You can turn these on in OneNote 2003 even now using Tools/Options – they really help if you care about structure.

We thought we were pretty clever when we designed that. But as some of us had feared, people came to think that they MUST write in the writing guides. Even if they knew they didn’t have to, writing outside the guides seemed wrong somehow, and it was distracting to cross a visual boundary you don’t have on paper. No matter how dim we made the guides, some people found them distracting, and the rest couldn’t even see them. And then the more we thought about it, the more this design seemed backward. People already knew where to write to continue a line or a list – they didn’t need us to tell them. In fact all these guides did was tell you where NOT to write if you DIDN”T want to continue a list or a line. And even that most people were pretty good at – they naturally wrote far enough away from existing stuff.

So we went back to fundamentals. What’s important with handwriting? Well, just getting the ink on the page is a big one (maybe 80% of the value at least is achieved just with that). Let’s make sure that is easy. Next might be searching the ink (say that’s 10% more). Next might be recognition to text (“reco”). About 8%? And then there is structure of the ink. Maybe that is 2%. Somehow we had got all excited as a team (me included) about the least important thing. It turns out that technology people do that all the time – its endemic.

We did what we could to fix up the ink to get the “80%” working as best we could. We also got the searching and reco of course. And there is even structure. But the whole experience is not the way we want it to be, so when we get a chance to really sit down and do it the way we want to, you’ll see great things.

BTW, one of the difficult things about finding bugs with ink (both in design and code) is being able to reproduce the problem in a developer’s office. Most bugs you can describe in a few steps, but we found very few bugs were being logged for ink even late in the product cycle. It turns out this was because the testers were embarrassed to write “draw some ink and it doesn’t work the way you expect” in the bug report. That’s not really reproducible, even with a picture of the results. So we implemented a way to capture the “wet” ink and send that directly to the developer so they could replay the ink being drawn. After awhile things got better. But then we noticed that the ink seemed to be behaving well in terms of code errors being few, but every person not familiar with “Scribbler” who tried to use it complained it was hard to use the ink. We couldn’t see it too well, but those fresh eyes could.

We organized events we called “ink-o-ramas”. We would collect about 10 people (actually Microsoft employee guinea pigs) who have never seen “Scribbler”, put them in a room, gave them Tablets, and asked them to take notes on a video taped lecture we played for them for an hour (and we gave them pizza – which didn’t go too well with the Tablets but we made do). Once people got used to Scribbler’s idiosyncratic ink interface, we had to change them out and get fresh “normal” people. We had 13 ink-o-ramas in total, each with a different focus near the end. We had one where we asked only native writers of East Asian languages to come, since those languages have a different interface for inking (Japanese, Korean, Simplified and Traditional Chinese). We got a lot of great, raw feedback, and we were able to see people having trouble in person – the devs were in the room too. By the time we finished all this, we had the ink working well enough to ship it, and it is what you see today. But we definitely know it can be better. Much better. We have plans…