[Sigia-l] Usability Testing comments from Giga

Tue Apr 1 20:31:20 EST 2003

> I know usability folks who test many other kinds of products with
> basically the same kinds of methods I use for testing computer
> applications.

Lyle I can assure you that I have worked through the years applying
Usability in areas beyond Computer-Human Interfaces.. The fact that in the
early years we called it HCI/CHI (depending on the site of the Atlantic), it
more had to do with the origins of the discipline...

> IA is much less exact and scientific than medicine or artillery.
> Without testing how do you know if you're hitting the mark or how to
> improve your aim?

Again, as with my original post.. so what kind of testing do IA folk do? If
you are an IA and work in the field how do you assure your clients that what
you propose to release will work?

Marios

-----Original Message-----
From: sigia-l-admin at asis.org [mailto:sigia-l-admin at asis.org]On Behalf Of
Lyle_Kantrovich at cargill.com
Sent: 02 April 2003 00:07
To: sigia-l at asis.org
Cc: Lyle_Kantrovich at cargill.com
Subject: RE: Re: [Sigia-l] Usability Testing comments from Giga

Too many people here try to argue for black or white, yet all I see is
grey.

Product Design (and hence IA), Medicine, Nautical Navigation, and
Artillery are inexact practices.  Some may be more scientific than
others, but still there are many variables to consider in being
effective in these inexact fields.

Usability testing (as Boniface said) is about risk management.  Test
plans should be driven by risk factors, business goals, user goals, and
past experience.  How you test a small workgroup intranet web site is
very different from the way you test a medical device (like a
resuscitator) or a NASA shuttle navigation system.  In reality, you
can't test everything - anyone who's done much testing knows that you
set priorities and create a test plan around the most important areas
of the product based on highest risk and most important goals.  (e.g.
if a user can't open the box, then they can't read the instruction
manual or actually use the product - you may need high confidence that
the box is easy enough to open.  Working on an online product?  Think
'login' instead of opening boxes.)

To Marios' question about what is a good test for an Information
Architecture, I'd point out that while HCI is focused primarily on
computer-based products, "usability" and User-Centered Design (UCD) are
not.  I know usability folks who test many other kinds of products with
basically the same kinds of methods I use for testing computer
applications.  Check out http://jthom.best.vwh.net/usability/general.htm

It seems obvious, but usability testing is primarily meant to assess
the USABILITY of a product.  There are lots of other kinds of testing
that can and should be done on different products.  Hopefully
Information Architects want their output/products to be usable - hence
usability tests should be very relevant to practicing IAs.  It's also
worth noting that usability testing is only one method of evaluating
the usability of a product.  See:
http://www.userdesign.com/usability_uem.html

There are lots of variables that can change over time: user experience
level, technology (OS, browser, etc.), language and semantics.  Hence
most products will benefit from testing - if nothing else, you validate
that what you "know" is still accurate and timely.

A product designer who doesn't test their product is like an artillery
crew without a spotter.  Unless someone can tell them where the round
from their gun falls, they don't know if they've hit their target or
not, and furthermore, they don't know if or how to adjust their aim.
They don't know if they need to take another shot at it.  Artillery is
very mathematical and 'scientific', yet they also lots of variables to
contend with.  Try this short article for an example.
http://www.kdhnews.com/military_art302.html

IA is much less exact and scientific than medicine or artillery.
Without testing how do you know if you're hitting the mark or how to
improve your aim?

Rolf Molich reminded me at CHI 2002 that there's a difference between
your opinion (theory) and what you've proven/tested within reason
(theory with direct evidence to support it).  Unless your theories
about your product's usability are grounded in some type of real
feedback from users actually using it, you're probably not standing on
very solid ground.  Usability testing is one form of feedback - no
more, no less.  How do you get your feedback if not through usability
testing?

Regards,

Lyle

----
Lyle Kantrovich
User Experience Architect
Cargill
http://www.cargill.com

Croc O' Lyle: personal commentary on usability, Information
Architecture, and web design
http://crocolyle.blogspot.com

"Simplicity is the ultimate sophistication."
	- Leonardo da Vinci

-----Original Message-----
From: listera at rcn.com [mailto:listera at rcn.com]
Sent: Sunday, March 30, 2003 11:03 PM
To: sigia-l at asis.org
Subject: Re: [Sigia-l] Usability Testing comments from Giga

"Jared M. Spool" wrote:

> While an experienced navigator can trust his gut to say that the ship
is on
> course, having instruments to verify position, speed, and conditions
gives
> everyone on board the confidence needed.

Speaking of one of my favorite subjects, for millennia perhaps the most
significant barrier to the discovery of our planet, the oceans, the
continents, etc., was not ignorance but the delusion of knowledge. The
Church gave us the 'ideal' map, the working seamen gave us portolanos,
harbor guides. Guess which one you'd consult if you were a captain of a
caravel off the cost of East Africa? I'm decidedly not against testing.
However...

> This is all about confidence -- trusting that we have the underlying
> knowledge, practices, and methodologies to produce reliably effective
> results every time.

...IA is not a profession where every result can be reliably and
predictably
'engineered' from a codified set of rules. I don't think that's even
needed.
In an incredibly fast moving platform of technology and development, we
(try
to) deliver efficient (and hopefully enjoyable) results.

Furthermore, I predict IA will never attain the 'scientific' status some
hope to achieve. IA is just as much a craft as medicine has been and is
today. As I provided earlier, we've come from days of autopsies being
done
by barbers to fantastic technological medical achievements. Medicine is
one
of the most highly regulated, licensed and tested professions around.
And
yet 50% of what doctors know today are invalid. So white lab coats are
not
enough and to the extent that they may obscure the forest for the trees
they
may even be detrimental in certain ways.

> Ziya asked an important question: does every decision need to be
tested.

The context I uttered what you paraphrased above was in reaction to an
assertion that someone would test what he was sure of, as well as what
he
was not. That's simply not attainable in the commercial world: there
just
isn't the money, resources, time and personnel for that. You'd have to
be in
the testing business to advocate that :-) That's why I gave the example
of a
doctor not testing everything during his practice, otherwise he'd have
no
practice in short order.

> Today, our confidence is weak.

I don't find that.

To me, the basic tenets of IA have been in practice for centuries. The
use
of the title may have gained currency with the birth of the web in
general,
but as I pointed out previously, what Mercator did with cartography or
what
Ray and Linnaeus did with the taxonomy of species was nothing less than
IA.
If you take a historical view, I think there's less reason for panic or
gloom. :-)

Ziya
Nullius in Verba

------------
When replying, please *trim your post* as much as possible.
*Plain text, please; NO Attachments

ASIST IA 03 Summit: Making Connections
http://www.asist-events.org/IASummit2003/

Searchable list archive:   http://www.info-arch.org/lists/sigia-l/
________________________________________
Sigia-l mailing list -- post to: Sigia-l at asis.org
Changes to subscription: http://mail.asis.org/mailman/listinfo/sigia-l

------------
When replying, please *trim your post* as much as possible.
*Plain text, please; NO Attachments

Searchable list archive:   http://www.info-arch.org/lists/sigia-l/
________________________________________
Sigia-l mailing list -- post to: Sigia-l at asis.org
Changes to subscription: http://mail.asis.org/mailman/listinfo/sigia-l
---
[This E-mail was scanned for viruses.]

---
[This E-mail was scanned for viruses.]