One of the main role of the researcher, is to provide quality, reliable
data to the product team. However, it is not quite as easy as it may
seem. Researchers, for lack of appropriate information or skill, may be
at risk of generating data lacking in quality. There are multiple
strategies that can be taken to ensure the highest quality of the data
being collected, including careful review of the research protocol and a
general skeptical approach to one’s research methodology. In this post,
I’ll introduce the underlying reason why a good researcher should
always initially cast a critical eye on the data he is being fed, and
apply this to his own data. I will then discuss some common threats to
good data collection I have encountered in recent weeks, and in
particular the threats of bad recruitment.
Coming out of my Ph.D., I was not quite prepared to become a user
experience researcher in the industry. While I was well prepared with
the research side of things, everything else was, and to a large extent
still remains, unknown. So as I am learning on the job my new role, I
thought I would share a thought or two about what I believe user
experience is, and some of the questions I still have.
First and foremost, I am currently building an understanding of the
role of UX research in the product development lifecycle, and how it
interacts with other sources of customer data, and in particular
To me, one of the key role of a user experience researcher is to
provide a fertile ground on which design ideas will flourish to become
nice and healthy designs. It is to provide the key understanding of the
users that will shape how systems are designed. This goes beyond
individual behaviors to look at the broader picture, the context in
which the system will be used in terms of technology, physical
environment, social context, work flows, etc.
For the skeptic reader, I would recommend reading the famous and inspirational paper by Heath et al.,
Collaboration and Control: Crisis Management and Multimedia Technology in London Underground Line Control Rooms.
It describes how by looking at the control room of the London Subway,
they uncovered key social interactions that would not have been
supported by the current plan for distributing the team in different
places. As is often the case, the initial understanding of the user for
the design decisions was too shallow to provide a holistic view of what
should be created and implemented.
For those interested to learn how my colleagues and I undergo grounding research, I will write a follow up post later.
As Paula has articulated recently, another critical aspect of user
experience research is to foster a new understanding of the usage of
technology, and to provide new perspectives on a particular area (in our
case, software development). By understanding the user and his
interaction with technology better, you can identify unexploited
opportunities for improving an experience, by supporting a behavior
subtle ways, by designing more flexibility in the product, or even by
sympathizing with the user and advocating for simpler, easier interface.
The question that remains is how to you go about inspiring the
product team? Some pieces of research are self explanatory. They make
anyone think of the design space from a new and refreshing perspective.
These findings are not necessarily validated (as in precise, realistic,
or generalizable, see Runkel and McGrath, Research on Human Behavior: A Systematic Guide to Method (1972)
for a neat explanation of validity compromises in research). Yet, their
goals is not to ground or validate design, but to challenge designers
(or engineers, product planners, managers, etc.) to think about the
design space in a new and interesting way, opening new avenues for
To conclude this already overlong post, it is time to discuss the
more traditional user experience job, validation. What I believe is
still often shortcut to usability, is in fact a set of method for
testing whether or not the design intent of a design (or a redesign) has
been met. It requires to understand what the goal of the (re)design
was, and to be able to operationalize (bless you) its success. This is
often a very tricky affair, as some people are not very agile at
articulating their design intent, and that often measuring the success
of this design can be an open-ended research problem in and on itself.
For example, in my thesis, I was tried to design communication systems
that made people feel closer to one another. The challenge I was faced
with was to determined whether or not I had succeeded with a reasonable
amount of confidence. Besides the obvious, asking the question to users
directly, I took on to operationalize closeness by assuming that if
people get closer, they get to know more about one another. The problem
of assessing increase in closeness became of problem of assessing
whether or not the users got to know more about one as they used the
system I had designed.
Not every validation study requires this sort of alternative
thinking. In fact, in many case, there are clear and measurable goals
that can be measured, such as time to complete task, number of
errors,etc. These studies, more common in usability, are a great way to
measure increase in effectiveness or productivity. They can rely on
measurable physical behaviors. They are a great way to double check that
you are really improving interaction and performances. They are usually
linked to the ability for users to understand the interaction model and
the information architecture, and to interact with it. However, they
can be misleading as there are many situations in which performance with
an interface is not the critical factor of the user experience. Take a
game for example, while users can be efficient using the interface, the
user is likely to really wants fun rather than efficiency.
Finally, there is always the polishing of the user experience, where
upon wrapping up the design, the designer wants to know if anything
major has been overlooked. By walking users through the design, the
researchers tries to capture critical misunderstanding or interaction
breakdowns that users encounter with the interface.
To be continued...
In the next episodes:
The Collaborative Side of User eXperience
The Convincing Side of User eXperience