[Sigia-l] Cluster analysis is dead - was Jakob wrong?

[Trent Mankelow]

Hi everyone. We're building a piece of software to replace EzSort
because we hate it. We've hired a very smart programmer and after a week
of "deep thinking" he's come back to us questioning the validity of
cluster analysis as a useful way of creating an information
architecture.   

(Before I get flamed - I totally agree that quantitative analysis needs
to be mixed with qual to make meaningful recommendations, but there is
no reason that the quantitative analysis shouldn't be of a high
quality.)

The earliest mention of the combination of card sorting and cluster
analysis 
that we have found is in Jakob Nielsen's article about his usability
work for SunWeb, published at http://www.useit.com/papers/sunweb/. This
outlines the attraction of using cluster analysis but doesn't talk about
limitations or do comparisions with other branches of maths, such as
graph theory.  

What's the history of this technique? How many people use it? Have we
been blindly applying cluster analysis without thought to its
limitations and alternatives?

Here's what Damon, our programmer wrote:

"There are two aspects that initially I find questionable:

1) When a similarity matrix is constructed, the process that creates the
matrix only captures information about cards that were grouped together.
If a person in the test decides they want to nest some cards below some
others, 
the parent child information is lost in the matrix construction.  That
makes 
the technique blind to the test subjects card hierarchies.

2) The similarity matrix and resulting clusters represent an average.
The card sorts may capture more than one fundamental 'mode', for example
the top level classifications could be year versus genre in a card sort
about movies.  If the modes are fundamentally opposed, such as year vs
genre, then the kind of average you would get out of cluster analysis
with all the sorts included could be a non representative average, ie
junk."

Thoughts and discussion most welcome!

Thanks,
Trent

Optimal Usability
:: People before technology ::
-----------------------------------------------------
email: trent.mankelow at optimalusability.com
website: www.optimalusability.com
phone: +64 4 381-4472 or  0800 USABLE
mobile: +64 21 389-494
-----------------------------------------------------