[Sigia-l] Automated User Testing

[mfg345 at aol.com]

I don't see this Automated User Testing as a way to replace our lab 
usability testing (or remote testing, which we are also exploring) . 
Nor do I want to replace our web analytics tools.

As simply as possible, what I'm looking for is something that fits into 
that vast chasm between web analytics (like Urchin) and lab (or remote) 
usability testing. Something that bridges the gap between the 
behavioral data that analytics provides (user took this path, user 
spent this much time on page) and exploration of intent that usability 
testing provides (why took this path, why spent so much time on page).

It may be true that "automated usability testing" is a myth, but is it 
because the space that such a service/tool falls in is not worth 
exploring, or because there simply isn't a good service/tool to do it? 
(also, it is worth noting that the two companies mentioned in my 
initial question do something vastly different than the company 
mentioned in your blog post). One such company that claims to have a 
"solution" for automated usability testing is Keynote (formerly 
Vividence), but I've not been particularly impressed with them in my 
experiences (especially in relation to their cost). What I'd really 
like is to be able to control the testing and analysis, while still 
reaching the number of users that their services provide, hence, my 
question.

Hope that clears things up, and of course, I'd also like to hear from 
anyone with experience with any such service/tool.

Michael

-----Original Message-----
From: Lyle Kantrovich <lyle.kantrovich at gmail.com>
To: mfg345 at aol.com <mfg345 at aol.com>
Cc: sigia-l at asis.org
Sent: Wed, 16 Nov 2005 10:41:27 -0600
Subject: Re: [Sigia-l] Automated User Testing

Michael,

"Automated usability testing" is a myth. If you're talking about some
other kind of "user testing", then can you clarify what you'd like to
test and what you want to get out of it?

There are many companies claiming to have "solutions" for this. I
blogged about one back in 2002, and it was one of my most popular
posts to date:

False prophets of Usability - Part 1
http://crocolyle.blogspot.com/archives/2002_02_03_crocolyle_archive.html
(Note that part 2 is still pending. :)

I'd be curious to hear from anyone on the list who's actually used
these kinds of products.

Back to your question: Why do you think you need to test "more users?"
If you're looking for ways to test with participants from a distance,
then you should look into "remote testing" methods, which lots of
folks are using and having good results with.

Lyle

--------------------------
Lyle Kantrovich
Blog: Croc O' Lyle
http://crocolyle.blogspot.com