[Sigia-l] User Test Cost - Does this sound reasonable?

[Jared M. Spool]

At 03:39 AM 5/24/2004 -0400, Listera wrote:
>Isn't there something inherently problematic about Developers attending
>usability tests or dealing with usability issues directly at all, unless
>specifically requested by Designers? (I can't imagine Designers being asked
>to attend, for example, a test of optimizing an app server function.)

Hi Ziya,

First, I think it is going to depend on the going definition of designer 
and developer. In my world, we use 'designer' very broadly -- a designer 
any member of the team that has an influence (either direct or indirect) on 
the outcome of the design. So, it would include influencers such as 
marketing, content producers, the management team, and even people we'd 
traditionally call 'developers'.

I can see your point about not asking designers to attend an optimization 
test, only because the information they'd gather from attending wouldn't 
change their design. If there's no benefit to attend, they shouldn't attend 
-- it's not a good use of resources.

However, developers often have direct and indirect influence over the user 
experience. They are making decisions about error messages, flow, and 
performance. All of these impact the user.

When we do testing, our goal is to ensure that all these decisions are as 
informed as possible. I've gone into a test with developers telling me that 
certain operations were 'technically impossible'. After they've had a 
chance to observe users struggle with the workarounds, the developers 
quickly proposed 'slight modifications' turned the impossibility into reality.

(This happened recently when the design team we were working with were told 
that a certain list box couldn't be populated because of an indexing issue 
-- it would just take too long to display the data and there was no way we 
could work around the problem. The team struggled with several workaround 
design solutions, none of them very satisfactory. As soon as the 
responsible developer sat in on some tests, he had several ideas on how to 
change the underlying application infrastructure to accomodate the original 
design.)

We don't conduct tests for the sake of running tests. We conduct tests to 
inform the design team, so they can make confident decisions that have a 
positive impact on the product. The audience of our results are the 
decision makers, whomever they might be.

Jared


Jared M. Spool        User Interface Engineering
http://www.uie.com    jspool at uie.com

Join us for UIE's extremely popular Roadshow event
UIE Advanced Techniques: http://www.uie.com/events/roadshow