[Sigia-l] Usability Testing

[Ziya Oz]

Bill Killam:

> The biggest issue with testing using an internal team is the conflict of
> interest inherent in being in that position.

If we were to take the notion of "internal = biased" to its natural
conclusion, we would have the absurdity of externally "validating" the
output of *everyone* who works for an organization, from the CEO to the
cleaning crew. Except for some legally mandated tasks (accounting,
compliance, etc.) we don't and really can't externally validate. But,
because the usability business targets designers, we are asked to believe
it's necessary to exclusively "validate" designers, although Sally at
marketing or Fred at supply chain can continue to make decisions on their
own, even if their decisions can be as/more significant than what designers
do for a product. 

> Project rarely (if ever) have time built into the schedule for iterations
> (though all of the user-centered design process, agile programming, and other
> models promote it).

If you're searching for conflict interest, look no further than that
statement. The project has no time for iterative design (fundamentally
important and necessary) but, miraculously, time *and* money has to be found
for external validation. This "usability tax" on project budgets, as framed
above, is the undeclared bias of that business.

> (In one case as part of an internal team, I was told by my manager to lower
> the classification of a safely issue to allow a system to be delivered based
> on the cost of repair.

If the manager is so ready to take a legally questionable/actionable path,
who's to say any amount of external "validation" can possibly dissuade him?

> External team are free (or at least freer) from these issues, though they risk
> alienating a client by being the bearer of bad news.

I hope you're not serious. I've been consulting for two decades on
medium-size to multi-million dollar projects for large companies. Anyone
who's been doing this for any length of time can easily observe that
especially the large-scale consulting factories are specifically managed to
maneuver projects towards billable directions where they (not the client)
can maximize their revenue. Talk about bias and conflict of interest!
Vendors have many uses (like those you cite below) but to claim that somehow
external usability businesses have inherently less bias than internal
designers, without context, is simply untrue.
 
> Three other main benefits of using an outside consultant include: (1)
> internal people are often more willing to listen to an outside (paid)
> consultant than an internal person who says the same thing (its human
> nature), (2) since (proper) testing is an activity that is not needed all
> the time, it can be more cost effective to hire a professional team when
> needed than to keep a staff on board full time for the occasions when they
> would be needed, and (3) seasoned outside consultants have more expose to
> alternate designs and design domains and can bring new perspectives into a
> project.

These are generally true.

Ziya
Nullius in Verba