[Sigia-l] Field Research request - summary responses

[gary_beck at vanguard.com]

Thanks to all contributors who replied (see end of posting).  Here are 
categorized responses: Benefits, Problems, Good Practices (by stage), and 
a few citations

Summary Benefits ----------------
Things you’d probably never see or hear about in lab interviews -– that 
can provide inspiration for features that can sway initial design 
concepts, inspire product differentiators, or motivate developing entire 
products in and of themselves
* The little black book of passwords
* The Post-Its around the PC
* The binders of related historical data kept in an adjacent room
* Session timeouts during online tasks
* The crowded work environment of their “real” workspace
  - The workspace that’s in the hall, with no space for booklets or 
welcome kits
  - The PC atop an armoire with a PC perched upon a pullout shelf
* Various and unusual PC configurations 
* The other hardware and software products used in conjunction with your 
product – and how well or not your product plays with them
* The workflow between using your and others’ products that the user 
navigates to accomplish various tasks
* User customization of any settings on the PC or with your product in 
particular (and why users have customized it in that particular manner)
* Other product collateral in the immediate workspace environment – and 
whether it’s the original documentation or something the user has 
customized (why and how?)
* User “stories” – with details often not heard in the lab and which are 
particularly useful in creating personas and personarios
* A better understanding of the users themselves, how they interact with 
others and their home environments, and how those impact their website 
interactions

The ability to bring back artifacts to share with the design team
* Printouts
* Hardcopy / softcopy documents
* Spreadsheets
* Laser engravings
* Photographs of items and work areas
* Videos of clients setting up for and performing tasks

This all hearkens to the phrase, “You had to be there” when trying to 
communicate to someone else about a special event that mere user verbal 
recollection doesn’t sufficiently capture


Summary Problems ----------------
* Travel difficulties, getting to various users’ homes
* Transcription task tedium – converting hours of video to text
* General documenting of the various details, even if not from video – the 
experience being so rich
* Not targeting (or sufficiently screening for) the proper user – 
sometimes the wrong (e.g., “professional,” “inarticulate,” “not right 
type”) client got through screening, and the effort was wasted
* Recruiting the desired individuals was time-consuming (where a 3rd-party 
recruiter was not used)
* Keeping users focused on interviews with children, dogs, TV, etc. as 
distracters (but also key data to gather)
* The challenge of communicating findings to the broader team – to those 
who weren’t able to accompany the researchers (sharing artifacts helps 
much in this effort)
* Not having the proper representation in the research team (i.e., bring 
someone from Design)
* Several cited experiencing no problems – neither before (gaining 
permission), during (getting users to communicate freely), nor after 
(system problems perhaps attributable to loaded software)
  - Note: Several indicated that their experience is that clients are not 
hesitant to discuss sensitive information such as financial matters.  The 
home environment was suggested as providing greater comfort to the users 
about sharing information 


Summary Good Practices ----------------
Before:
* Prepare a research agenda that clearly outlines the general questions 
you seek to answer as well as the specific things that you are going to 
ask
* Test your research plan with a few participants (perhaps local, or 
acquaintances) before you head out into the field. 
* Come equipped to capture details of the visit through various means
  - USB stick for downloading files of interest (screen captures too) – 
also useful for seeing other apps running simultaneously 
  - Recording with a camcorder 
  - Digital camera
  - Audio recorder
* Be prepared with a plan B if the target behavior is computer-based, 
because the user’s computer may malfunction
* Prepare a non-disclosure (Confidentiality) form for the user to sign, 
and include an opt-out clause
* If researching for a 3rd party, consider not stressing that fact – at 
least initially -- as it may bias user behavior and comments
* Even with a signature, ask the client again before doing any recording 
or capturing
* Be certain client is made aware of compensation for their participation 
(examples ranged from bring-along lunch to $250/hr)
* Perform these type studies early in the design process
* Get sponsor buy-in on the target users
* Choose researchers with skill, compassion, and experience – and who do 
not have too much investment in a particular finding or outcome
* Choose clients carefully, screening for those who meet key criteria and 
are judged to be willing to share
* Consider using at least 2 people on visits (for safety and liability 
reasons) – more than 2 can be intimidating for the user

During:
* Note: Be prepared to forsake recording media if the client is squeamish 
to gather valid data
* Don’t interrupt the flow of a task – keep your data-collection agenda in 
mind, but don’t let it interfere with natural procedures
* Don’t “fix” things for the user, even if it looks simple
* Don't let "scientific validity" of this subjective exercise negatively 
impact the client's or subject's experience - it's counterproductive 
* Make sure you are getting everything out of your research protocol as 
much as possible and be willing to change it as you progress.

After:
* Share artifacts as much as possible with the broader team to help 
refocus on target users and away from design-team personal preferences
* Document findings and produce the final report as quickly as possible – 
both for better remembering details and for producing impact
* Present the report in a respectful, friendly, and compassionate voice
* Stress the benefits (ROI) of the study in the report, so the sponsor is 
rewarded for having taken the risk of proceeding with the field study
* Share key insights in the context of a story that a participant was 
telling you in a particular context -- not in a bullet-pointed list.. 
  - Use pictures and video clips to illustrate the context. 
  - Well-crafted personas are a great way of capturing this kind of 
research insight because they help to instill empathy and can be quite 
easy to disseminate in an organization.
* Illustrate in proposed product design how the gathered information 
impacts the proposed design (attesting to the value of the field research)
* In order to help continue organizational support of future field 
research, be selective in its application

Citations ----------------
* H. Beyer & K. Holtzblatt, Contextual Design: Defining Customer-Centered 
Systems.  San Diego: Academic Press, 1998
* Leonard and Rayport’s Harvard Business Review article,  "Spark 
Innovation through Empathic Design, 1998
* Gilmore’s article in Interactions, "Understanding and Overcoming 
Resistance to Ethnographic Design Research, 2002
* Chinese Home Site Visits - Tips & Hints - 
http://www.apogeehk.com/articles/tips_for_chinese_home_visits.html
* Nokia (Jan Chipchase) blog on conducting field research for Nokia 
http://www.janchipchase.com/blog/mt-search.cgi?tag=design%20research&blog_id=1
* Yahoo’s web-based solicitation for participating in User Research 
http://promo.yahoo.com/user_research/ 


Contributors ----------------
* Adaptive Path – anonymous researcher
* Baxevanis, Alexander
* Deaton, Mary
* DiSilvestro, Tony
* Fischer, Henning
* Foltmer, Tina
* Hinton, Andrew
* Kakar, Priyanka
* King, Ellen
* Merholz, Peter
* Monahan, Kelly
* Price, Susan
* Soucy, Kyle Pero
* Szuc, Daniel
* Virzi, Bob
* Warfel, Todd Zaki
* Wu, David

Gary
x37588

----------------------------------------------------------------------
CONFIDENTIALITY STATEMENT. The information contained in this e-mail message, including attachments, is the confidential information of, and/or is the property of, Vanguard. The information is intended for use solely by the individual or entity named in the message. If you are not an intended recipient or you received this in error, then any review, printing, copying, or distribution of any such information is prohibited, and please notify the sender immediately by reply e-mail and then delete this e-mail from your system.