I was ranting to Mr. Chen today
about how he has it all easy with his blog. (Raymond and I work on the same team,
he just dresses better). See, Raymond is a developer, an extremely accomplished and
intelligent one at that, whereas I am but a humble UI guy. While he can talk to old
and technical details, I can't. Fundamentally, I'm wired to rant about, and write
about, user experience in software design.
That is harder*, in case you aren't following.
I've got to think a bit - make the blog interesting, provide value - and really, get
some actual use out of it for me. I think part of that is doing my job to contribute
to developer education on UI/UX (which I believe is terrible today) by discussing
the subject in-depth. Sit back - this is going to be a constant theme. Feel free,
of course, to seed me with new ones.
I get asked a fair amount about how we do usability
work at Microsoft. Generally, there’s genuine
interest in what we do and how. But there is also a lot of interest in how people
themselves can do usability work – or often, misconceptions about why they can’t.
Side note: There’s not enough room (or
time) here to go into the specifics of all the things you can do, or the theory behind
them. Instead I’m going to just talk a bit about one method you can use on the cheap,
some experiences I’ve had with it, and then we’ll come back to the subject over the
course of this blog.
Of the simple methods out there to conduct usability work, one of the key tools is
the use of paper prototypes. A paper prototype is essentially a bunch of print outs
of different states of UI. All you need are:
print outs of every step of the UI you want to test
usability subject (we’ll talk about recruiting these in another post) – the more,
the merrier, but you aren’t really going for breadth here
set of ‘tasks’ you want the user to accomplish
time to sit down with them
For example, let’s say you were testing the tasks in the My Pictures folder. You would
make a print out for different states of the window, and perhaps different ones for
each task you might click on (what it would look like after you had clicked) and so
on and so forth through the entire flow you would want to test. When you sit down
with your usability participant, you would quite literally have them look at the “screen,”
click on items with their finger, and place the new “screen” down on top.
This is incredibly cheap – you need a willing subject, but defining the tasks are
simple (what is it you want users to do with your UI), and creating the paper print
outs is much cheaper than building the software first and fixing it – it doesn’t have
to look fantastic. I find that even basic diagrams in Visio can do the trick – especially
at the early stages of design (before you really want to commit a ton of resources
in a particular direction).
As you let the subject walk through the UI (trying to accomplish the tasks you’ve
set out for them), you want them to think out loud. This literally means saying things
like “well, I think I’m going to click on this because it says ‘FooBar’ and that sounds
like what I’m looking for” – don’t worry if this sounds goofy. You also want to make
sure to capture what the user is doing at each stage and what they say. You can have
a friend record the thoughts and actions (What is the flow of UI they walked through,
and what did they say at each juncture?) tape it on camera (which is great to review
later), or simply record it yourself (harder to do, but if you are good at note taking
you can handle fairly easily).
What you gain from this are two main points of insight. The first is, “What the heck
does this say to the user?” By having the subjects think out loud and seeing what
they do, you can draw the connections between what they perceive and what they actually
want. Some points will be obvious, such as too much heavy text on the screen, or buttons
that aren’t close enough to be seen as part of the same decision point, etc.
Once during a usability session where we were walking a user through some screens
we’d drawn up (around a picture experience), we were trying to get them to find the
task to print some pictures. They read everything they saw, including the task that
said, “Print out pictures.” However, that task was somewhere in the middle of the
list of things they read (And they would consistently read everything on the screen
all the time, trying to be extra careful) and they would miss it. We have the user
on tape, reading the list out loud, including this very task, and they weren’t able
to complete the task!
We were flustered – nothing could be easier than this task. Was this because the user
was illiterate? No, the issue was really that the UI was too busy, we had too many
things going on to absorb. It wasn’t because they couldn’t read, it’s because they
were too busy** to think about it. This didn’t take fancy mockups and expensive
usability engineers to figure out, either…
The second major insight that you can gain is “How successful is the user using this
UI?” The importance here is on the task completion – were they able to do what you
wanted, and how so? While the first point revolves around the story the users were
told, the second talks about effectiveness. You might claim that only the second point
is relevant, but the first is equally important because you want to set yourself up
for future tasks to be easy to perform, and for you to be able to build on something
that makes sense to the user.
There – you’ve successfully figured out how to test some UI on the cheap, get a good
understanding of what’s working, and perhaps most importantly, why it is working.
One additional reason causes me to point to paper prototypes as a great way to do
some basic usability testing: users tend to
be more critical with paper. We’ve seen this time and time again in the lab. When
users sit down in front of a computer for a usability test, they often do not have
much knowledge on what goes into creating software – except that it’s hard and expensive.
They are getting something in exchange for being there (such as free software or being
paid, etc) and really want to a) tell you what you want to hear and b) not cause you
to have to do any big rework or throw anything away.
This isn’t true for all users, but it is likely true for the majority. What
you’ll find is they simply refuse to be all that critical*** – they don’t want to
be the cause of your having to redo something. This is a subconscious thing as well,
so it’s not easily combated. The beauty, however, is that it doesn’t really come into
effect with paper prototypes. Those are clearly disposable and cheap – and users will
have absolutely no problem telling you what they think. You’d be surprised by the
There’s some other great parts about testing actual UI or experiences directly on
a computer – such as the interface (mouse mechanics are harder than pointing with
a finger, etc), as well as the depth (it’s easier to have a program running to test
all the possible outcomes than create printouts for everything), and so on.
That’s testing with paper prototypes. There’s more depth to it than the space I’ve
devoted, but it is enough to get started with in case you are out there. It’s even
useful to use on yourself – talk out loud as you walk through UI (away from a computer
with just the print outs). Walk through a couple flows, think about what the user
might see (That you don’t). In-depth variants of this are called “cognitive walkthroughs”
but we’ll get to that another time.
* To be clear, I mean harder for me, not Raymond, because I’m simply not as bright.
** Users being too busy is one of the basic rules of UI. Users aren’t stupid, they
aren’t blind, they aren’t incapable: they are just too busy to learn your UI. Humility
is good for user experience.
** This isn’t to say they are dumb or don’t say critical things, it just tends to
be gentler, and they tend to do things like blame themselves more often.