[Sigia-l] A Brief History... (was Design by testing)

[Peter Merholz]

Ash wrote:

> Although Human Factors

Can I just say how much I hate the phrase "human factors," and the degree to
which it is indicative of the narrow-mindedness of the field? It drips of a
certain Taylorist quality, as if you are meant to find out the "human
factor" in some larger system. ("Users" suffers a different, though
similar-ish problem).

> Unfortunately, many people in these professions today don't even know of
the history of their
> area of specialisation (or more accurately these days - generalisation.
It seems UEs and IAs
> both try and do the same job - most of which is not supposed to be their
job - but that's
> another discussion).

A fairly simple one. There aren't enough of us to do it right, so of course
those who practice or or the other will find their responsibilities spilling
over into "other" disciplines.

> And yes Bill, this has also led to what you describe as "user-directed
design" - a
> bastardisation (through no fault of their own) of user centred design.
People see themselves
> as 'qualified' to work in this space after doing a 2 day course covering
card-sorting,
> scenarios, paper prototyping, and 'speak aloud' protocol - simply because
they don't know that
> anything else exists.

Aha! And hear the elitism of "trained" user-centered designers begins to
rear its head. I would argue that such folks *are* qualified to begin
working in this space. You've got to start somewhere. And if you're some
project manager drone in an organization that doesn't recognize "usability,"
and you want to build a better product, then more power to you -- get out,
take a class, read a book, and do what you can.

> Unfortunately, this idea of such a simple technique was a tumble-on effect
of Neilsen's '94
> paper - "Guerrilla HCI: Using Discount Usability Engineering to Penetrate
the Intimidation
> Barrier," in which he proposed all that was needed was scenarios,
simplified thinking aloud,
> and heuristic evaluation to meet basic needs.  This paper was the impetus
on one hand for
> popularising usability (excellent!), but on the other (as a tangential
effect) for diluting
> its effectiveness.

Diluting? How?

> Neilsen (1994) prefaced his paper with "I will focus on achieving "the
good" with respect to
> having some usability engineering work performed, even though the methods
needed to achieve
> this result are definitely not "the best" method and will not give perfect
results."

Which is all that ought to be asked in any real-world situation. Satisficing
goes a long way. Striving for perfection will needlessly bog you down.

> Today, these oversimplified techniques are taught en masse with no
background provided as to
> (among a plethora of other things) why that particular technique should be
employed, the
> importance of participant selection (and how to derive a representative
sample), how to
> observe and analyse (as opposed to just taking what a participant says
verbatim as a
> requirement) - even what controlled, uncontrolled and confounding
variables are!

The second you begin dealing with controlled, uncontrolled, and confounding
variables, the second you're entering the realm of science, which, I'd
argue, is not very interesting to the majority of this group. The
user-centered design we practice did not come out of academia, but out of
engineering, where you try to do the best you can given limited resources
(be it time, money, or people). It's not an attempt to scientifically
validate a solution, but simply to get the best product out there.

This was all better said in the most recent issue of Interactions magazine
by Dennis Wixon, who works in Microsoft's games division.

I wrote about Wixon's article here:
http://www.peterme.com/archives/000122.html

I consider it easily the most important usability article I've read all year
(if not for a few years), because Dennis gets beyond the pseudo-science to
recognize that, hey, we're all just trying to do the best job we have given
the resources, and you can take your "number of users" metrics and your
"controlling for variables" tests and shove them, because they're getting in
my way of getting a good product out on time and on budget.

--peter