[Sigia-l] card sort order effects

[donna at maadmob.net]

I don't understand what you are working on that you would be both card 
sorting and usability testing at once (assuming that when you say 'at once' 
you mean on the same day not a few months apart.)

It sounds vaguely like you have an existing site and you are trying to 
determine whether the site structure is usable. If this is the case, I have 
found that usability testing (eg. traditional at computer, scenario-based, 
speak aloud testing) is not a great way to test site structure. In many sites 
there are so many information-retrieval scenarios that it is difficult to test 
them in this type of usability test. I have, however, found that this type of 
usability test is great for testing that participants can get around the site 
well, and that the screen elements are usable.

So, assuming this is the right scenario, I'm not sure what you would gain 
out of card sorting at the same time. Are you giving participants content 
that is on the existing site, and asking them to sort it into groups and label 
them?

Again, assuming I have your scenario right:
- if you do it before testing, people will have their own groupings and labels 
in mind which will impact on how they navigate the site. If their groupings 
are different to yours they are likely to feel inadequate - after all, they will be 
thinking of you as at least some type of expert and will assume that your 
structure is right, which means their grouping must be wrong.
- if you do it after the usability test (not forgetting that they will be at least a 
little tired), they will be either influenced by the structure and labelling that 
they have just used and will reflect it, or assume that you are asking for 
improvements so will try not to do what the site already does.

Either way, I can't imagine that the results will be useful. The bias will be 
too strong.

If I absolutely had to do something like this, I'd somehow make sure that 
my usability test scenarios and my content in the card sort did not overlap 
at all. Perhaps you can tell them that you only want to test a small part of 
the site, and get them to organise content for the other part. And I'd still 
split them into 2 groups, change the sequence and watch for impacts (but 
with such small groups it will be difficult to detect anyway)

Hope this makes sense. I may have completely misinterpreted ;)

Donna


On 1 Aug 2003 at 17:14, Martin, Chris C. wrote:

> Hello all,
>  
> I'm involved with a usability test and card sorting exercise. For both we
> have to use the same set of nine participants, so we have to decide whether
> to do the card sort before or after the usability test.
>