[Sigia-l] User Test Cost - Does this sound reasonable?

[Stew Dean]

At 02:45 30/05/2004, Pabini Gabriel-Petit wrote:
>This discussion has moved on since I last had time to contribute, so I'll be
>brief.
>
>BJ Cook wrote:
> > > Our IT manager doesn't believe in usability and testing, he believes in
>blind
> > > deployment of his software and apps.
>
>Todd R.Warfel responded:
>That can usually be resolved by either having the person observing some
> > actual usability testing (behind the mirror or in another room via
>closed circuit TV - you don't want the participant hearing the IT
>manager yelling how stupid they are for not being able to figure out
>his software)
> > If they're not available for that, then showing them video vignettes
> > will typically help.
>
>[PGP] You're making a big assumption here, Todd. Where management isn't
>friendly toward user-centered design, it's sometimes impossible to get a
>budget for any usability testing. Such companies certainly don't have
>usability labs with 2-way mirrors or closed-circuit TV.
>
>And if that doesn't work, then putting some actual
> > dollar figures on the costs associated with poor usability will
> > typically do the trick - since managers traditionally deal with
> > budgets, playing the numbers game is a way to break into their sense of
> > reason.
>
>[PGP] Any suggestions for where to get this data?


You want ROI figures?

There is a chapter on this in the polar bear book (Information Architecture 
for the World Wide Web - Luis Rosenfeld, Peter Moreville).  I won't 
reproduce the figures as it's a good book but will say that you could focus 
on time saving of someone working internally.  For example one of the most 
common tasks for any intranet (if that's what you are designing) is simply 
trying to find something - something people who do information architecture 
excel in. So you can say a good user experience can save the average 
employee ten minutes a day.  Take the average wage, the number of people in 
the company etc and you have some kind of figure.

Now I know you then have to prove you can save ten minutes. Which is down 
to your original problem.

I think the emphasis should not be on testing - it should be on research 
and the first myth of research is it has to be expensive. If you are 
already part of a company, designing a customer facing system and there are 
customer service people in the company you already have a great source of 
information about your uses. You can gather information from them that can 
show how the blind process doesn't work, you can get great insight into the 
problems with the existing system from them. For Intranets, even easier, 
you have a whole company of users who will happily tell you about their 
problems.

Overall I try and paint a picture that if you are designing anything for 
anyone how are you going to know what they want without asking them. If you 
are designing a suit you measure a person, get their preference on 
materials etc and then you do a fitting (akin to user testing). You can't 
design a suit based on common sense - why would any IT project be different?

So user research can be inexpensive if you can find the right people and 
requires only your time. Convincing managers can usually be done via return 
on investment figures. If they can spend X and make X times 5 and this can 
be demonstrated then even the most cynical would think again. Easier said 
than done but at times you've got to really stick by your guns, doing any 
design without research is, after all, just a bit insane.

Stewart Dean