[Sigia-l] RE: UCD and Usability Testing (was: Usability testing into the dustbin?)

[Mitchell Gass]

At 07:43 PM 5/2/2003 -0400, Boniface Lau wrote:
>"Usability by design" means usability should be designed-in, not 
>tested-in...A team staffed with people capable of usable design tends to 
>produce systems containing relatively few usability problems before 
>testing...In contrast, a team staffed with people relying on usability 
>testing tends to produce systems filled with usability problems before 
>testing...It is a sloppy way of creating usable systems.

What I'm reacting to is a blanket rejection of usability testing, which I 
now see is based on a profound misunderstanding of testing's proper use.

No one who is at all sophisticated about user-centered design would argue 
that usability testing at the end of product development should be the 
first time that developers observe and interact with users. Design should 
be informed by a thorough understanding of users, their activities, and the 
context of those activities. For almost all designs, the best time to gain 
that understanding is at the very beginning of the development process, 
before any resources are spent on implementation.

One way I explain usability testing is with a circus analogy: if design is 
a hire-wire act, usability testing is the safety net. You never want to use 
the safety net. You never try anything unless you're confident it will 
work. But if your judgement about what will work isn't perfect, or if you 
simply make a mistake, the safety net gives you another chance.

Designers who never do anything daring may not need a safety net. If your 
designs are all small variations on previous solutions in well-understood 
domains, it's like walking a wire two feet off the ground. But when 
anything about a design is fundamentally new, there's risk. Usability 
testing is a way to minimize that risk and support innovation.

My advice to design teams is always to test what they believe will work. 
This does not mean that what's being tested has to be finished or fully 
functional; it's possible and often valuable to test early prototypes and 
design elements. But what's being tested should be solid: there's no point 
in having users discover that something doesn't work when the designers 
already know it won't work or aren't confident about it.

All of this gets back to some fundamental questions about design:

- Does design benefit from being user-centered?

- Should designers interact with users during development and, if so, when 
and how?

- Should design be an iterative process and, if so, when is it best to iterate?

- Should designers evaluate the success of designs before releasing them 
and, if so, how?

I have some extreme views on these: I'm a proponent of participatory design 
who believes that users should be on design teams, and my preferred design 
methods are like those used in The Bridge

   http://www.participatorydesign.com/BridgeIntro.pdf

that go a step beyond waterfall and iterative approaches to a "whitewater" 
approach where assumptions and decisions are continuously reevaluated, 
changed, and refined by stakeholders during design sessions in response to 
new information and new insights. Users participate in the sessions, and 
user testing - of task analyses, object models, and prototypes - occurs 
throughout the design process.

Mitchell Gass
uLab | PDA: Learning from Users | Designing with Users
Berkeley, CA 94707 USA
+1 510 525-6864 voice
+1 510 525-4246 fax
http://www.participatorydesign.com/