[Sigia-l] Questioning common test scripting

[InfoArchitect]

Livia wrote: 
> A positive message ("help us test X") works better than a negative 
> one ("we are not testing you"). 

Exactly my point.  From a cognitive psychology POV, using a negative 
emphasis in this context is not acceptable, yet most of the scripts I 
have come across (even some from cognitive engineering labs) contain 
those words "We are not testing you".  I'm just curious as to how 
they: 
1. Started being used; and 
2. Became a defacto standard without someone questioning the impact 
such a statement may cause.

Using a redundant negative to emphasise such a point only serves to 
create anxiety in the subject, and I can't see why it seems to be so 
rife in the scripts that I have come across.

Ziya wrote:
> How would you [1] know if the participant is at ease (or, for that
> matter, petrified, scornful, melancholic, adversarial, intimidated,
> aroused, etc.)?
> [1] As a non-medical person.

There are a number of methodologies for assessing emotional states in 
cognitive psychology.  In fact, Hudlicker (2000) cites that there are 
over 200 anxiety instruments alone, many focusing on specific types of 
anxieties (e.g., mathematics, social, etc.).  

I actually tried to place a disclaimer in my original post so as not 
to divert the discussion down this seemingly typical path of 
tangential, semantic arguments that seems to occur.  Obviously, I did 
not cover all eventualities.  

The point of this discussion is not how a professional can assess such 
states, but why it seems to be common practise to use a script that I 
hypothesise, would introduce a feeling of anxiety for participants, 
when the object is to study typical responses that would occur in a 
natural, relaxed environment.  

I am wondering if the advent and subsequent popularity of ‘web 
usability’, is diluting scientific rigor in human 
factors/HCI/cognitive psychology evaluation techniques.

There has been much debate recently over the validity of data produced 
by contemporary testing, but has anyone questioned:
The methods (not just if statatistical significance is practically 
attainable, but are you using the correct type of test?); 
The means (should this test be performed in a lab, in the field, etc.  
Are the participants representative of a specified profile, or are 
they simply "someone who uses the web"?); or 
The qualifications/backgrounds of those performing the tests (do they 
actually understand correct experimental rigor - know the difference 
between an independent and dependent variable and the need to 
eliminate confounding factors - or were they just taught how to carry 
out some standard tests)?  

Regards,

Ash Donaldson
User Experience Designer  

----------------
Get your free email address from www.ourbrisbane.com