[Sigia-l] Learnability and its impact on testing

[Eric Scheid]

On 22/9/03 12:36 PM, "Tania" <tania_peakusability at optusnet.com.au> wrote:

> I am still not convinced how to measure and taken into account the
> learning variable as it significantly varies for different people.

Theoretically, once you do factor out the learning variable then the best
fit line for the test subject's success rate would be flat (ie. no slope)
over the course of the exercise. You could try weighting later questions
such that the best fit line levels out. This does assume that there is no
bias in relative difficulty between the early part of the exercise and the
later part (ie. easy questions first, then the curly ones). Randomising the
sequence of questions would help, I think.

> 1) putting suspected problem navigation areas first,
> 2) mixing scenarios around a lot so there weren't a lot of scenarios for
> one area of the site grouped together,
> 3) mixing the order of scenarios presented to different test
> participants (so the same scenarios were not always at the end);
> 4) noting and disregarding some of the results that were obviously
> skewed by the participants memory

You could also try avoiding having questions in overlapping areas, such that
what they learn on earlier questions cannot be used in later questions. This
might make the test set very short though :-(

e.