[Sigia-l] SUMMARY: Equipment recommendations for a Mobile Usability Lab

Thu Jan 16 09:49:34 EST 2003

Thanks to everyone who sent feedback.  Here is the
high-level summary:

1. Buy a packaged mobile usability lab! Will simplify
the process, and labs are developed to be easily
portable, and fit into nice carrying cases.  

Recommended vendors included:
* User Works
http://www.userworks.com/default.asp?page=products&sub=
* Twin View
http://www.neenanpartners.com/twinview/index.htm
* Ovo Studios 
http://ovostudios.com/

2. Assemble your own lab, with:
* Laptop computer (could also connect lab to
respondent’s computer)
* Screen-recording software (though this has
limitations)
* Video camera (digital or high-8)
* Tripod (for camera)
* Monitor (could also connect to monitor available on
site)
* Keyboard (could also connect to keyboard available
on site)
* Microphone (to capture user comments)
* Tape recorder (to capture sound away from computer)
* Portable printer (for printing last-minute changes
to screeners or paper prototypes)
* Lots of cable (to connect computer to viewing room)
* Small TV (to set up in viewing room)
* Double VCR (for copying tapes)
* Video mixer (for creating split screen of computer
and user)
* VHS tapes
* Digital tapes
* Carrying cases for all equipment

Phew!  Sounds like a lot of work.  :)

Let me know if you have additional comments.  Here are
the individual responses I received:

--- Bill Killam <bkillam at user-centereddesign.com>
wrote:

I realize you said your budget is small, but there are
vendors that provide portable usability lab solutions
already configured for picture in picture, remote or
in room use, data logging, etc.  We have two of them
at our offices and wouldn't consider building one
ourselves even though we have the technical savvy to
do so.  Yes, you can throw some equipment together and
get by, but I haven't found it to be cost effective.  

The labs we use are fully portable and can be placed
in overhead on planes for safe transport.  And our lab
vendor even has developed adaptors we use for testing
paper prototypes (with picture in picture) as well as
testing PDAs, cell phones and other portable devices,
and even for conducting man-in-motion (sic) activities
(or meeting and focus groups).  Unless you want to get
into the lab business, building even a rudimentary one
could open a can of worms. Then there's the support
and maintenance issue.  

BTW, I would not consider your own laptop a necessity.
 You'd have to connect into someone else's network or
phone line to get access to the internet - something
not everyone is comfortable with.  And many tests we
do are early in the design phase, sometimes behind
people firewalls, and not always on the internet. 
Your situation may be different.  Then there's the
issue of accessibility and the need to load and
configure different types of adaptive technologies.  

Though we can and do use own laptops on occasions, we
often use our labs connected to the participants own
computer.  It's non-intrusive and saves a lot of work.
And screen capture software has a number of severe
limitations for usability testing.  It also risky for
data loss and has some hidden costs. (Believe me, I've
been down this path many times.) 

--- Laura Scheirer Quinn <Laura.Quinn at IntraSphere.com>
wrote:    

I've found it extremely useful (and cheap!) to test
with users in their own environment.  I post a
prototype or prototype to the web under security,
login through the security for the user, and then let
them interact with the site on their own computer
(this is obviously more feasible with business
applications, where you can find a number of potential
users in one physical location, than specialized
consumer applications).  Interestingly, I've found
people much more critical and better able to focus on
how the site fits in with their tasks in their own
environment, on their own computer (presumably because
they're more comfortable...).

I generally go with someone else, and one person
conducts, and the other takes detailed notes on the
user's task flow, what they click on, what they say,
places where they seem to get confused.  We'll audio
tape sometimes as well (though these are really hard
to follow without video-- I use them primarily only to
pull out good quotes that I remember or we've noted as
occurring).  

It's a particularly guerilla technique-- I find it
very effective to do research for my own benefit, to
understand what's working well and not so well.  I
think what you really lose in not having video is the
ability to play clips for skeptical decision makers to
convince others that weren't there.  I don't find it
much of a loss for myself, as it's so time consuming
to try to go back through all the tape and do
additional analysis-- I find I rarely have time for
this.  It's a struggle to try to get buy in to make
time for any testing to start with.  

--- Alfred <alfred at thunderstick.com> wrote:  

http://www.alienware.com/main/gaming_main_mobile.asp  

Alienware makes great looking machines - originally
for the gamer market which means good graphics. The
Area-51m comes with a backpack, and if this is meant
for client sites, these computers certainly dress to
impress :) You might call and ask what they have to
meet your needs - they are used to people packing up
their computers for LAN parties.

--- Stephen Holmes <sholmes at topladder.com.au> wrote:

Why an external keyboard as a requirement? You are
testing mobile apps on a laptop and who packs an
external keyboard when they are "mobile"?

Laptops position different keys such as "f1 to f15",
"home" and other navigation aids and of course there
is no separate number keypad, so you have to test
mobile apps in a totally mobile situation and you have
to design your app for ease of use on a laptop
keyboard, NOT external keyboard (or monitor for that
matter).

I'd even suggest that testing is done in a airplane
seat and fold down table as well if you are developing
a business app. (grin)

Things to remember when designing mobile apps:

* Keep mousing to a minimum or provide keyboard
alternatives to navigation,

* If you also wish to migrate to Pocket PC or Palm,
start the design process with all platform limitations
in mind from the very start.

* Try to keep the structure as flat as possible on
Palm-like apps since there is less screen area
available for navigation aids such as bread crumbs or
drill-down navigation bars.

--- R.T. de Boer <deboer at campeador.com> wrote: 

The microphones to capture the user comments. Sound is
extremely important! 

--- Scott Weiss <sweiss at usableproducts.com> wrote:    

We have a great portable lab--but it's more of a
portable full-size lab than a portable lab. I found
the portable labs required too many sacrifices.

Our lab could be duplicated for about $5k-$7.5k in one
shopping trip or a thorough order from B&H online. It
fits in two rolling cases that each weigh about 70#.
However, we include at least three sets of 50' cable,
a full-size computer, speakers, keyboard, and an LCD
projector... As well as a double-VCR, a video mixer,
digital video recorder... And we carry 14 VHS tapes
and 7 digital tapes.  

We rent televisions and computer monitors at each
destination, though the cost is ridiculous. It's
cheaper to order one new and leave it there.  The
portable setups all require sacrifices. We didn't
sacrifice anything except size and weight. It also
kept cost down.  

Digital video cameras are overrated, by the way. The
quality is the same or less than a high-8 camera, and
we always mix that source with the scan converted
computer output, so a digital camera would be
silly--we don't record that image by itself.  You
could also go with Ovo Studios' solution, but they
probably sell it at a real premium. 

--- John Fullerton <JFULLERT at lib-gw.tamu.edu> wrote:  

Recently I found the following quote:

I always have a DV Cam pointed at the screen during
testing, so I just connect it to my powerbook via
firewire. Then I use quicktime broadcaster
(http://www.apple.com/quicktime/products/broadcaster/)
to stream it. It literally took 3 minutes to get it
set up.

--- Donna Maurer <donna at maadmob.net> wrote: 

Don't forget a microphone. You will need some type of
deck to mix the images from screen and camera.  I have
just ordered a mobile lab too. I didn't worry about
the computer equipment as I plug into client's
equipment (I'm doing it internally, so this is fine).
I also ordered a small TV and lots of cabling so I can
set up the developers in the next room, watching on
screen.  Ask your vendor if they can provide good
sturdy cases for the equipment, as you will be moving
it a lot.  

--- Michael Sichmeller <michael.sichmeller at usbank.com>
wrote:  

Though I haven't actually implemented this product, I
have had it demonstrated a few times. There are other
similar solutions on the market but I like what Twin
View offers. John Neenan, the head of the company is
working with Human Factors International to improve
his product as well. 

http://www.neenanpartners.com/

--- Cinnamon Melchor <cmelch at sapient.com> wrote: 

You might want to look at the stuff User Works has for
rent, either with an eye toward renting it (if you're
in the DC/Baltimore/Philly area) or using it as a
shopping list.  

http://www.userworks.com/default.asp?page=products&sub=

--- Chris Shafer <cshafer at modemmedia.com> wrote:  

You should consider adding tape recorder for any pre
and post interviews that might be done away from the
computer and a portable printer for any screeners you
might need to produce on the spot. Printer would also
be helpful for doing paper based prototyping, which
would make sense if you're under pressure to test more
frequently and on shorter notice. 

__________________________________________________
Do you Yahoo!?
Yahoo! Mail Plus - Powerful. Affordable. Sign up now.
http://mailplus.yahoo.com