[Sigia-l] Usability Testing

[Bill Killam]

>> The biggest issue with testing using an internal team is the conflict of
>> interest inherent in being in that position.

>If we were to take the notion of "internal = biased" to its natural
>conclusion, we would have the absurdity of externally "validating" the

Nowhere in my statement do I use the word biased or make any inference to
it.

>But, because the usability business targets designers, we are asked 
>to believe it's necessary to exclusively "validate" designers...

Evaluating designers?  Wow, that quite an opinion to take or at least a
parapraxsis (Freudian slip). You think that evaluating a design is the same
as evaluating the designer  Any designer with their ego that tied into their
design is precisely why THEY cannot be objective testers.

>, although Sally at marketing or Fred at supply chain can continue to make
decisions on their
>own, even if their decisions can be as/more significant than what designers
>do for a product. 

Hardly the same, and ever Fred and Sally have checks and balances in the
jobs - just different ones.

>> Project rarely (if ever) have time built into the schedule for
iterations...

>If you're searching for conflict interest, look no further than that
>statement. The project has no time for iterative design (fundamentally
>important and necessary) but, miraculously, time *and* money has to be
>found
>for external validation. This "usability tax" on project budgets, as framed
>above, is the undeclared bias of that business.

I'm not searching for conflict of interest and there is no conflict of
interest in the statement.  I suspect you mean there is a discrepancy in the
statement.   However, that is not the case either.  The question asked was
one of internal versus external testing so that presupposes that testing IS
planned for.

> (In one case as part of an internal team, I was told by my manager to
lower
> the classification of a safely issue to allow a system to be delivered
based
> on the cost of repair.

>If the manager is so ready to take a legally questionable/actionable path,
>who's to say any amount of external "validation" can possibly dissuade him?

I didn't say an external firm would dissuade him (though you do agree to
that positions - at least in general - later.)  I said that internal
pressure from a boss is why the problem with an internal test team unless
they have full management support and protection.  Since this was a request
from my boss, my job with the company was at risk if I did not comply.  That
defines the conflict of interest.

> External teams are free (or at least freer) from these issues, though they
risk
> alienating a client by being the bearer of bad news.

>I hope you're not serious. 

I am absolutely serious.

>I've been consulting for two decades on
>medium-size to multi-million dollar projects for large companies. Anyone
>who's been doing this for any length of time can easily observe that
>especially the large-scale consulting factories are specifically managed to
>maneuver projects towards billable directions where they (not the client)
>can maximize their revenue. Talk about bias and conflict of interest!
>Vendors have many uses (like those you cite below) but to claim that
>somehow
>external usability businesses have inherently less bias than internal
>designers, without context, is simply untrue.

Thanks for the credentials. I have been consulting for nearly 3 decades on
medium-size to multi-million dollar projects for large companies.  I never
said there was not a similar conflict of interest for some consultancies.
And I do agree its more prevalent with the larger ones.  But the fact that
they recreate the very conflict I'm talking about is proving my point, not
yours.

> Three other main benefits of using an outside consultant include: (1)
> internal people are often more willing to listen to an outside (paid)
> consultant than an internal person who says the same thing (its human
> nature), (2) since (proper) testing is an activity that is not needed all
> the time, it can be more cost effective to hire a professional team when
> needed than to keep a staff on board full time for the occasions when they
> would be needed, and (3) seasoned outside consultants have more expose to
> alternate designs and design domains and can bring new perspectives into a
> project.

>These are generally true.

Glad we agree (generally) on at least some points.

Bill