[Sigia-l] Usability Testing

[Ziya Oz]

Bill Killam:

> You might want to get independent validation of at specific milestones in
> the design process (e.g., at the completion of conceptual design, after the
> completion of a new IA, after preliminary design, after detailed design,
> after operational prototype) depending on your confidence in the design or
> if unanswered research questions exist.

Designers should test *whenever* there arises a need, not when some
artificial schedule milestone demands it. That's deliverables culture.

Sorry I forgot who it was now, but someone also mentioned user "feedback"
with respect to design. Extremely slippery slope. Very hard to manage. I am
far far more comfortable with user/task observation than user "feedback."

I have been using this slide for a few years now in my strategy
presentations, and I believe I sent a copy to someone on this list last
summer:

"I skate to where the puck in going to be, not where it has been."

I fell out of my chair Steve Jobs thus quoted Wayne Gretzky in his iPhone
keynote. That too should be tattooed on everyone, even remotely involved in
product design.

Users are not there to tell/insinuate/divine/feed/direct/validate our
designs. We're there to observe what they do and learn. Users are as biased
as they get. Whether it's 5 or 12 or 16 or 37.5, it's very difficult to
untangle their "input/feedback" for anything approaching "independent
validation."

I come in to do UI surgery when everything else has failed on big projects.
In virtually every case there are existing user tests/validation. And in
virtually every one of those cases I've seen, there are way too many
interdependent variables being "tested" to derive any rational/useful
conclusions. And clients have a difficult time figuring out what to believe
when I point out the absence of any credible baseline. Unfortunately,
"everything else is not equal" and designing and conducting even half-way
scientific tests do take time and money, which very few clients are willing
to spend at adequate levels. So for far too often than I care to see, a lot
of variables get all jumbled up to get "validated." While I can appreciate
the resource pressure folks are under, the results are really misleading, to
be charitable.


Ziya
Nullius in Verba