This is a slightly different version of the material that we presented at CHI this year. We've got some related material and examples that I can post if there is interest. The authors are:

Rick Spencer, Monty Hammontree, and Donna Wallace

 

 

Abstract

Analysis of existing product design and development methods revealed an underlying common framework consisting of three components and three principles. The components are (1) a multifaceted view of experience, (2) a set of goal driven phases, and (3) a repository of best practices for navigating those phases. The principles are (1) team-based design, (2) divergence and convergence, and (3) alignment. To apply the Experience Engineering Framework, , teams should construct a UCD process tailored for each version of the product by utilizing the right components and principles rather than always relying on one design method.

Introduction

A team of usability engineers and product designers in Microsoft’s Developer Tools Division systematically analyzed numerous User Centered Design (UCD) methods. This analysis revealed an underlying common framework for all such methods consisting of three components and three principles.  The three components that all strong user-centered design approaches have in common are: (1) A multi-faceted view of “experience,” (2) a phased model approach, and (3) a practices repository. The three common principles are: (1) A team-based approach to design, (2) Fostering divergence and then convergence of ideas, and (3) An active approach to alignment of teams and phases.  We named the combination of these components and principles the Experience Engineering Framework (EEF).

Component 1 – A Multi-faceted View of Experience

A complete UCD method has a rich view of experience defined by some multi-faceted model. Practitioners should choose such a model to suit the needs of the team. These models of experience fall into one of four different types: (1) Causal and Predictive Models, (2) Structured Planning Models, (3) Categorization Models, and (4) Role Based Models. The following sections provide examples of each.

Causal and Predictive Models

Causal and predictive models attempt to help practitioners understand how different facets of experience explain or predict user behavior or preferences.

The Technology Adoption Model (TAM) demonstrates how five variables of experience (usage, satisfaction, utility, usability, and desirability) can predict satisfaction, usage, and therefore business value. According to TAM, usage is the best predictor of both user and business value And the primary driver of usage is satisfaction.  Usefulness, Usability, and Desirability all load on satisfaction. Usability also loads on perceptions of usefulness.

See attached figure: TAM.jpg, adapted from Davis, et al[1].

Structured Planning Models

Structured planning models provide a framework for practitioners to gather data.

Holtzblatt and Beyer defined “the 5 faces of work:”  the flow model, the sequence model, the artifact model, the cultural model, and the physical model. The use of these models ensured that interviewers were focused on the appropriate facets of the experiences they were trying to understand[2].

Categorization Models

The purpose of Categorization models is to help practitioners organize and understand existing data.

Andy Cargile, UX manager in the Microsoft Hardware group, extended Cooper’s “Users, Business, Technology” Venn diagram[3] to create a classification scheme with three categories of needs: stated, latent, and opportunistic. Existing user data points are placed into one of these categories, enriching a team’s understanding of different kinds of needs.

See attached figure: carg.jpg, a needs categorization model that supports three facets of understanding user needs.

Role-based Models

Role-based models help clarify the various perspectives and practices of the divergent disciplines involved in product development efforts.

Morrogh[4] provides a list of stakeholder perspectives:  sponsor/client, user, designer, programmer/engineer,, and product manager. A variation in this is the disciplines typically involved in creating a web site:  usability, information design, interaction design, and programming/engineering.

Component 2 – A Phased model approach

All UCD methods have some notion of phases or steps.  For example, Cato[5] lists several phase models that are available in the literature, and settled on a three-phase model (discover, design, use) that included several subphases within the design phase. Other methods range from three[6] phases to eleven[7]. Perhaps the most well known phase model is Morogh’s fivephase model, Discover, Define, Concept, Design, Implement. Even methods whose main audience is computer programmers define similar phases[8].

Phases are implicitly or explicitly defined by their goals. A group of interaction designers described their role in terms of four problems that they solve[9]. Three of these problems—understanding people, defining what to build, and defining how to build it—can be thought of as phase goals.

By focusing on the goals of a method, we derived a common five-phase model into which all other methods can be projected. This five phase model underlies all other phase models.

Phase

Goal

Understand

Create a shared understanding of users, business, and technology. Collect inspirational user data

Envision

Create a shared vision of user value, and a shared vision of flow

Specify

Create great interaction designs and cross-product consistency

Implement

Implement great visual design, user-centered design iterations, and UX defect management

Maintain

Drive adoption and support users

Table: Common phase model, with five invariant phases and related goals.

Practitioners should use the common phase model to identify the strengths and weaknesses of existing UCD practices, and to create an Experience Engineering Plan that includes the right practices for each phase.

Phase

Contextual Design

Understand

Work Modeling

·          Contextual Interviews

·          Team interpretation sessions

·          Flow models

·          Cultural models

·          Sequence models

·          Artifact models

Consolidation

·          Affinity diagramming

·          Consolidated work models

Envision

Work Redesign

·          Vision (scenarios, storyboards)

·          User environment design

·          Paper prototyping

·          User iterations

Specify

Prioritize and Object-Oriented Design

·          Modified QFD prioritization matrix

·          Use cases

·          Object models

Implement

<empty>

Maintain

<empty>

Table. The Contextual Design methodology phases and deliverables mapped to the Common Phase Model. Notice that it lacks any guidance for managing the Implement or Maintain phase, but provide rich tools for navigating the early phases.

Component 3 - Best Practices Repository

Practitioners should have a toolkit of best practices, with a good understanding of when and where to apply each practice. Every method, or book of methods, is essentially a repository of best practices, and by using the Common Phase Model, we can see each practice. But it is still difficult to select the right methods and to know what contextual factors should be considered to determine the best practice for a particular problem and time.

Cost and time are two of the main constraints used to determine the appropriate best practice. Understanding the trade-off between these two factors and the “usability problems found” has been covered extensively in the usability literature.

However, we continue to find these two simple factors to be inadequate for describing how a senior UX professional actually selects the right methods. Both Wixon[10] and Seffah[11] point out that the subtleties of successful application of different methods is not currently communicated in the literature, and they both suggest a “case study” approach for sharing this critical contextual information.

Our vision is to create a repository of best practices, where each practice is linked to the goals from the Common Phase Model to which it applies, and annotated with contextual information to help UX specialists map each practice to their current context.  Mayhew takes steps toward this goal by listing numerous usability methods in relation to the phase in which they are best used, and pivoting around project complexity as a contextual selection factor.

Principle 1 – A Team–based Approach to Design

The value of teamwork to product design has been described by numerous eminent practitioners.  Multi-disciplinary teams with a facilitator have proven to be the most successful.

A facilitator ensures that all team members contribute and deliver their best work. Furthermore, facilitators drive the process and ensure that the phases and public deliverables are completed.

Because design teams benefit from divergent view points, they should be mult-disciplinary with developers, project managers, user researchers, designers, QA staff, writers, etc. This heterogeneity has two benefits. First, every problem and potential solution is examined from multiple vantage points. Secondly, participants have different skill sets, frames of reference, personality characteristics, and thinking styles.  This diversity helps prevent groupthink and usually results in better final products.

Principle 2 – Fostering divergence and Convergence of Ideas

Practitioners should help teams first diverge  and then converge in their thinking.

“Divergent thinking is the ability to generate many possible responses…to a challenge[12].” Divergence is necessary for teams to identify the best possible ideas, rather than settling for the first workable idea. Divergent methods elicit and incorporate as many ideas as possible.

Convergent thinking and tools are used to reach consensus on the best ideas to carry forward. Successful convergence is typically marked by the creation of a document or set of documents. The notions of divergence and convergence are frequently represented in an “accordion” model that maps divergent and convergent activities onto a phase model[13].

See attached figure: accordian.jpg, a typical “accordion model” representing divergence and convergence through phases, including typical activities for diverging and converging in each phase.

Principle 3 – An Active Approach to Alignment of Teams and Phases

Phase Alignment

Phase alignment is about keeping teams focused on what was learned in the previous phases. As plans are impacted by external events, teams should revisit outputs from previous phases to help them understand and adapt. Outputs may need to be updated or changed.

Team Alignment

Team alignment enables teams to work together and understand the goals of other teams. Typically teams use documentation that can be shared across teams, and updated to reflect changes.

For example, the Voice of the Customer and related methods target alignment between different disciplines and phase alignment by sharing four “quality houses.” One quality house links user needs and perceptions (Understand) with target design attributes (Envision), another links the design attributes with design ideas (Specify), the third links the design ideas to implementation decisions (Specify), and a fourth one links implementation decisions to production planning (Implement)[14].

These four documents are shared across teams to drive a common product understanding.



[1] Davis, F. D, Bagozzi, R. P., & Warshaw P.R. (1989). User Acceptance of Computer Technology: A Comparison of Two Theoretical Models. Management Science, Vol. 35, no. 8. pp. 982 – 1003.

[2] Holtzblatt and Beyer reference

[3] Inmates are running the asylum

[4] Morrogh, Earl. Information Architecture: An Emerging 21st Century Profession. 2003. Pearson Education, Upper Saddle River, NJ.

[5] Cato, John. (2001) User-Centered Web Design. Pearson Education Limited. London, UK.

[6] Mayhew, Deborah, J. The Usability Engineering Lifecycle: a practitioner’s handbook for user interface design. (1999) Academic Press, San Diego, CA

[7] Nielson, J. (1993) Usability Engineering. Academic Press, Inc. San Diego, CA.

[8] Fowler, Martine, Kendall, Scott. (1997) UML Distilled: Applying the Standard Object Model Language. Addison Wesley, Reading, MA

[9] Rettig, Mark & Korman, Jonathan & Olsen, George & Erin & Bomback, Jennifer, & Andy, & Irwin, Terry, & Grefe, Rick & Terry. Problems solved by experience design: AIGA Experience Design work session results (2001).

[10] Dennis Wixon, Evaluating usability methods: why the current literature fails the practitioner, interactions, v.10 n.4, July + August 2003

[11] Ahmed Seffah , Eduard Metzker, The obstacles and myths of usability and software engineering, Communications of the ACM, v.47 n.12, p.71-76, December 2004

[12] (Isaksen, Dorval & Treffinger 1994).

[13] Pugh, S. (1990) Total Design: Integrated Methods for Successful Product Engineering.Reading, MA. Addison-Wesley.

[14] Griffin, Abbie and John R. Hauser (1993), "The Voice of the Customer," Marketing Science, vol. 12, No. 1, (Winter), 1-27.