[Sigia-l] Ethnography - using actors in fieldstudies

[Ziya Oz]

Eric Reiss:

> this IS very much like mystery shopping - and the results can be very good.

What's the benchmark?

> Stuff like this is critical if you're going to make this interview
> believeable. 

IOW, the benchmark comes down to the performance of the actor.

Under this setup, you could conceivably replace the FA also and have two
actors interact with each other based on scripts. That might be great
theater but it ain't a 'field study' unless you want to really stretch the
meaning of that phrase.

I don't know if you actually watched FAs in action but good ones, like all
good salespeople, are very attuned to verbal cues and body language. They
have trained themselves over countless such interviews to parse for such
cues and redirect the conversation accordingly. The actor's performance,
whether on or off script, will certainly have material impact on that
two-way conversation, at which point you're conducting a play not a field
study.

I do architecture/interface/design autopsy and surgery for a living. When I
look at projects where 'usability tests' were conducted, virtually every
single one of them will have an extremely predictable source of failure: the
inability to control variables in multivariate tests. Variables have
cumulative and interdependent impact on each other and the whole. It's the
easiest urge in the world to say, well, we don't have time, budget or the
analytical insight to properly pair them.

In this case, you are letting 2 of the 3 variables get corrupted by relying
on the performance of an actor. That's not analytically sound testing
practice and your data is bound to be suspect.

If you want to get a general idea of what FAs do, none of this matters. But
if you want to unearth deeper and specific truths, you're in make-believe
land.

-- 
Ziya

It depends.
If it didn't, you'd be out of a job.