[Sigia-l] Questioning common test scripting

[InfoArchitect]

Donna wrote: 
> Now I have to think more about how to do this...

To convey to people that you are not testing them, there are many 
approaches.  Removing the words "not" and "test" negates the need to 
autonomically or subconsciously introduce the notion that the user is 
being tested.  Try saying something along the lines of: 
"Today we'll be looking at a new design for <'x' type of product>.  Of 
course, there's going to be some flaws in that design, so the purpose 
of today’s exercise is to find any areas that may cause confusion or 
difficulty for people."

Ziya wrote: 
> "Established" by whom?

The cognitive sciences.  Human Factors, the study of human-system 
interaction, was around long before the Internet.  The area of study 
was first officially 'christened' in 1946, but its history spans much 
further back than that.  I believe that you’ll find that from this 
area of cognitive research came the many forms of usability testing 
and user-centred design methodologies.
 
> If you go through the archives of this list, you'll discover that 
> there are some people (many not coincidentally in the testing
> business) who argue that the future of IA is in white lab 
> coats, "scientific" methods, usability"engineering", etc. They argue
> essentially for design by testing. In case you haven't noticed :-) I
> have a beef with that notion.

Ziya, Mitchell and others will probably despise me for this, but I 
come from the point of view where yes, usability testing IS a 
scientific process (i.e. a cognitive psychology based verification 
method).  To enforce scientific rigor, we try not to introduce 
extraneous variables or confounding factors in any methods, even 
subjective ones (such as Likert-scale questionnaires), thus the origin 
of this thread.

I agree wholeheartedly with Ziya on one point.  Usability testing does 
not create good design, nor does it even create intuitive or usable 
design.  It was never designed to create.  It's merely a method to 
verify design assumptions (read: "hypotheses"). 
  
Knowledge of your intended audience; the creation of primary and 
secondary personas; task analysis; contextual inquiry; competitive 
analysis; structured questionnaires; open-ended interviews; scenario 
design; conceptual walkthroughs; GOMS analysis; hierarchical task 
analysis; colour theory; layout theory; task priority; heuristic 
reviews; ethnography; cognitive loading analysis; performance 
requirements; etc, etc, ad nauseum.  These are all requirements, tools 
and methods that help to create a good, usable design.  

In a user-centred design methodology, there are a number of different 
types of usability testing that can be used.  Using Verbal (Think 
Aloud) Protocol at every stage seems to be another popular trend 
amongst Web professionals, which is why I again pose the question:  Do 
we use the right method at the right time?  Does everyone know which 
method to use and when?  Does everyone know what is available?  No, 
I'm not advocating a 'white coat brigade', nor do I argue for 'design 
by testing' - merely the correct use of the tools and methods.  

For those who are new to the area, usability tests can be broken up 
into four types and many methods.  The main types are exploratory, 
assessment, validation and comparison.  The method used should be 
dependent on the goal of the test and whether the test is performed 
during the formative or summative phases.  They can be subjective or 
objective, quantitative or qualitative.  Testing methods include: 
Coaching Method; Co-discovery learning (better than think aloud in 
many cases if you want to know what a user is thinking); Performance 
Measurement; Question-asking Protocol; Remote Testing; Retrospective 
Testing; Shadowing Method; Teaching Method; and Think Aloud Protocol.  
When performed correctly, these are all scientific methods with a 
hypothesis, dependent and independent variables, as well as possible 
extraneous variables and confounding factors that must be controlled. 

Since the advent of the Internet and the popularity of usability, 
there has been a massive influx of people who claim to be 'as 
qualified as you can be' because they've perused a few books and there 
are no degrees in Usability or Information Architecture.  Fair enough, 
it's a fledgling industry and these graphic designers and business 
analysts will eventually find their way to the cognitive or library 
sciences.  In the meantime, however, I have noticed a trend toward the 
bastardisation of these scientific methods.  There are debates raging 
that they don't work.  To borrow one of Ziya's metaphors, it's like 
having your best friend operate on your Mother because he's a personal 
trainer that has read a couple of books on surgery, and then claim the 
surgical methods don't work because she died during the operation. ;)  

Best regards,

Ash Donaldson
User Experience Designer


----------------
Get your free email address from www.ourbrisbane.com