I was ranting to Mr. Chen today about how he has it all easy with his blog. (Raymond and I work on the same team, he just dresses better). See, Raymond is a developer, an extremely accomplished and intelligent one at that, whereas I am but a humble UI guy. While he can talk to old and technical details, I can't. Fundamentally, I'm wired to rant about, and write about, user experience in software design.

That is harder*, in case you aren't following.

I've got to think a bit - make the blog interesting, provide value - and really, get some actual use out of it for me. I think part of that is doing my job to contribute to developer education on UI/UX (which I believe is terrible today) by discussing the subject in-depth. Sit back - this is going to be a constant theme. Feel free, of course, to seed me with new ones.

I get asked a fair amount about how we do usability work at Microsoft. Generally, there’s genuine interest in what we do and how. But there is also a lot of interest in how people themselves can do usability work – or often, misconceptions about why they can’t.

Side note: There’s not enough room (or time) here to go into the specifics of all the things you can do, or the theory behind them. Instead I’m going to just talk a bit about one method you can use on the cheap, some experiences I’ve had with it, and then we’ll come back to the subject over the course of this blog.

Of the simple methods out there to conduct usability work, one of the key tools is the use of paper prototypes. A paper prototype is essentially a bunch of print outs of different states of UI. All you need are:

·        Paper print outs of every step of the UI you want to test

·        A usability subject (we’ll talk about recruiting these in another post) – the more, the merrier, but you aren’t really going for breadth here

·        A set of ‘tasks’ you want the user to accomplish

·        Some time to sit down with them

For example, let’s say you were testing the tasks in the My Pictures folder. You would make a print out for different states of the window, and perhaps different ones for each task you might click on (what it would look like after you had clicked) and so on and so forth through the entire flow you would want to test. When you sit down with your usability participant, you would quite literally have them look at the “screen,” click on items with their finger, and place the new “screen” down on top.

This is incredibly cheap – you need a willing subject, but defining the tasks are simple (what is it you want users to do with your UI), and creating the paper print outs is much cheaper than building the software first and fixing it – it doesn’t have to look fantastic. I find that even basic diagrams in Visio can do the trick – especially at the early stages of design (before you really want to commit a ton of resources in a particular direction).

As you let the subject walk through the UI (trying to accomplish the tasks you’ve set out for them), you want them to think out loud. This literally means saying things like “well, I think I’m going to click on this because it says ‘FooBar’ and that sounds like what I’m looking for” – don’t worry if this sounds goofy. You also want to make sure to capture what the user is doing at each stage and what they say. You can have a friend record the thoughts and actions (What is the flow of UI they walked through, and what did they say at each juncture?) tape it on camera (which is great to review later), or simply record it yourself (harder to do, but if you are good at note taking you can handle fairly easily).

What you gain from this are two main points of insight. The first is, “What the heck does this say to the user?” By having the subjects think out loud and seeing what they do, you can draw the connections between what they perceive and what they actually want. Some points will be obvious, such as too much heavy text on the screen, or buttons that aren’t close enough to be seen as part of the same decision point, etc.

Once during a usability session where we were walking a user through some screens we’d drawn up (around a picture experience), we were trying to get them to find the task to print some pictures. They read everything they saw, including the task that said, “Print out pictures.” However, that task was somewhere in the middle of the list of things they read (And they would consistently read everything on the screen all the time, trying to be extra careful) and they would miss it. We have the user on tape, reading the list out loud, including this very task, and they weren’t able to complete the task!

We were flustered – nothing could be easier than this task. Was this because the user was illiterate? No, the issue was really that the UI was too busy, we had too many things going on to absorb. It wasn’t because they couldn’t read, it’s because they were too busy** to think about it. This didn’t take fancy mockups and expensive usability engineers to figure out, either…

The second major insight that you can gain is “How successful is the user using this UI?” The importance here is on the task completion – were they able to do what you wanted, and how so? While the first point revolves around the story the users were told, the second talks about effectiveness. You might claim that only the second point is relevant, but the first is equally important because you want to set yourself up for future tasks to be easy to perform, and for you to be able to build on something that makes sense to the user.

There – you’ve successfully figured out how to test some UI on the cheap, get a good understanding of what’s working, and perhaps most importantly, why it is working.

One additional reason causes me to point to paper prototypes as a great way to do some basic usability testing: users tend to be more critical with paper. We’ve seen this time and time again in the lab. When users sit down in front of a computer for a usability test, they often do not have much knowledge on what goes into creating software – except that it’s hard and expensive. They are getting something in exchange for being there (such as free software or being paid, etc) and really want to a) tell you what you want to hear and b) not cause you to have to do any big rework or throw anything away.

This isn’t true for all users, but it is likely true for the majority. What you’ll find is they simply refuse to be all that critical*** – they don’t want to be the cause of your having to redo something. This is a subconscious thing as well, so it’s not easily combated. The beauty, however, is that it doesn’t really come into effect with paper prototypes. Those are clearly disposable and cheap – and users will have absolutely no problem telling you what they think. You’d be surprised by the difference.

There’s some other great parts about testing actual UI or experiences directly on a computer – such as the interface (mouse mechanics are harder than pointing with a finger, etc), as well as the depth (it’s easier to have a program running to test all the possible outcomes than create printouts for everything), and so on.

That’s testing with paper prototypes. There’s more depth to it than the space I’ve devoted, but it is enough to get started with in case you are out there. It’s even useful to use on yourself – talk out loud as you walk through UI (away from a computer with just the print outs). Walk through a couple flows, think about what the user might see (That you don’t). In-depth variants of this are called “cognitive walkthroughs” but we’ll get to that another time.


* To be clear, I mean harder for me, not Raymond, because I’m simply not as bright.

** Users being too busy is one of the basic rules of UI. Users aren’t stupid, they aren’t blind, they aren’t incapable: they are just too busy to learn your UI. Humility is good for user experience.

** This isn’t to say they are dumb or don’t say critical things, it just tends to be gentler, and they tend to do things like blame themselves more often.