[Sigia-l] Usability Testing

[Will Parker]

On Jan 22, 2007, at 2:36 PM, Donna Maurer wrote:
> Oh, I wasn't implying that I should be the only person who works on a
> design. I truly believe in teamwork, and real user involvement*. I  
> just
> don't believe that usability testing provides the best possible input.
> Particularly usability testing from someone uninvolved...

Oh, I agree. See my recent comments regarding levels of commitment in  
consultants.

> And as for the number of people involved in a usability test, more
> really is better. I don't take individual comments and suggestions  
> from
> participants - I want to see the main issues and patterns. I like
> conflicting evidence - gives me things to think about. But that is not
> possible with 4 participants.

'But DON NORMAN says to use FIIIVE!'   	};->

Actually, the smallest group of usability test subjects I've  
personally seen was six, and that was because the PM had made it  
clear he wasn't listening to advice from anyone who couldn't get him  
fired. More usual at Microsoft and a few other places I've been was  
eight to sixteen test subjects, with sixteen being the usual desired  
size and ten being the mode on how many actually showed up.

> * I also don't think usability testing is real user involvement.

Usability testing is QA, not design input. The time for 'real user  
involvement' comes *before* design, when product management  
(including designers and planners, but lord help us please not  
Marketing) goes out in the field and interviews a reasonably large  
sample of presumptive users of the product about how they'd like to  
conduct their lives around the aching void that the product will  
presumably fill and the designer can presumably deliver.

That's the real validation stage, whether it's formally called that  
or not. Do we really know the shape of the hole we think we can make  
money and/or fix a problem by filling? Are we crazy, or are other  
people aware of the same hole? Is the hole, which we think is there,  
worth filling, yes or no?

Yes, you can attempt to validate the design by usability testing  
after you've worked up some prototypes, but the serious mistakes come  
from poor assumptions stemming from misreading the design domain, and  
those are much harder to fix than any disjointed design flaws you'll  
find in usability testing or public betas.

"Good" examples of bad designs that couldn't have been fixed by  
usability testing:

	- The Apple 'Cube Mac'
	- The Microsoft Zune
	- The Ford Edsel
	- The Sony ATRAC file format

I'd say that each of these represent a serious mis-read of the 'hole'  
-- the design domain -- that needed to be filled to accomplish the  
company's goals.

- Will
Will Parker
wparker at ChannelingDesign.com

"The only people who value your specialist knowledge are the ones who  
already have it." - William Tozier