[Sigia-l] The CogTool Project

[Steven Pautz]

While it would definitely be great to discover how experienced users
will interact with a system simply by running a simulation, tools such
as GOMS and ACT don't tell us *how* users will behave -- they only
predict performance data based on existing predictions+assumptions
about what the users will do.

If you tell it actions, it will tell you ideal task time.

Personally, I believe this makes it useful for the same sorts of
situations where a "classic" usability test with a specific,
predefined task and a stopwatch is useful: demonstrating the potential
for improvement to project stakeholders. For actually delivering
improvement, however, I'd say it's largely irrelevant, as it can't
provide any insight or information beyond the number of seconds
expected for an expert user to complete a specifically-delineated task
-- and we all know how easily-misinterpreted and -misused that data
can be.

Disclaimer: I haven't used this specific tool, although as near as I
can tell, it seems to just be a GUI sitting on top of an existing
model....
----------------------------------------
Steven Pautz
Graduate Student
Human Factors Psychology, Clemson University




(Some additional information/thoughts, for anyone who's REALLY bored
and hasn't heard of this before.)

Some of the more advanced models (eg, ACT-R) can be very valuable (and
quite valid/reliable) for cognitive psychologists interested in
internal cognitive mechanisms -- it's accurate enough that one can
show a genuine correspondence between modeled cognitive approaches and
real-world task times -- in essence, it can be used to suggest
validity (or falseness) of a specific model of internal cognition for
a specific task. For that purpose, it's arguably one of the best tools
around.

For design, however, it provides almost no information at all. It
can't generate, suggest, or inspire ideas or potential solutions. It
can only simulate whatever user actions you specifically tell it to
do: it can't read or misread, can't get confused or frustrated, can't
go off course or make mistakes, can't smile or frown, and can't make
comments or suggestions.

It's not supposed to, though, since those things are [presumably]
unlikely to occur for experienced users. Presumably, experienced users
will jump through the task almost mindlessly -- there's no pausing to
figure out where to go or what to do. Cognitive modeling tools can
predict task times rather well for this kind of behavior -- but,
again, they're only returning task time; to my knowledge nobody's even
attempting to get at the user experience.

If "reduce the time needed for expert users to do ABC" is the sole
design goal, cognitive modeling is a Godsend. If the boss believes
that reducing task time for expert users on task ABC will increase
profits, but doesn't believe it can be done, cognitive modeling can
provide data faster and less expensively than real users. For anything
else, I don't believe it will provide any justifiable benefit, either
in resource savings or in the eventual quality of the design solution.


On 1/18/07, Ziya Oz <listera at earthlink.net> wrote:
> Anyone had any experience with this?
>
> Cognitive Performance Modeling
>
> Designers want to know how users will interact with their systems, but they
> often do not have the time or resources to test their designs with actual
> users. This is especially true for systems intended for trained users. It
> can be expensive to train users with a prototype, but unskilled users do not
> interact with a product in the same way as a skilled user. Cognitive
> performance modeling aims to solve this problem by predicting how a skilled
> user will interact with a system. Simulation and analysis take the place of
> expensive training and testing.
>
> <http://www.cs.cmu.edu/~bej/cogtool/index.html>
>
> ----
> Ziya
>
> Client <- Designer -> User
>
>
> ------------
> IA Summit 2007:  Enriching IA
> Rich Information, Rich Interaction, Rich Relationships
> March 22-26, 2007, Las Vegas, NV
> www.iasummit.org
> -----
> When replying, please *trim your post* as much as possible.
> *Plain text, please; NO Attachments
>
> Searchable Archive at http://www.info-arch.org/lists/sigia-l/
>
>
>
>
> ________________________________________
> Sigia-l mailing list -- post to: Sigia-l at asis.org
> Changes to subscription: http://mail.asis.org/mailman/listinfo/sigia-l
>