[Sigia-l] Ethnography - using actors in fieldstudies

[Ken Matthews]

If your goal is to measure the performance of the FA, then this is exactly
like mystery shopping, which can be used to a company's advantage.  No, as
Ziya points out, you probably won't publish your research in a journal - but
that's okay. 

As long as you are aware of the uncontrollable variables the results can
still provide valuable feedback on FA performance given a particular
situation.  You won't be able to widely generalize your results but that
does not mean they are useless.

I suggest, though, that you not warn the FA at all, otherwise you increase
the chance that the FA realizes someone is an actor if the actor's
performance is somewhat off.

Ken


On 11/28/07 4:11 AM, "Ziya Oz" <listera at earthlink.net> wrote:

> Eric Reiss:
> 
>> this IS very much like mystery shopping - and the results can be very good.
> 
> What's the benchmark?
> 
>> Stuff like this is critical if you're going to make this interview
>> believeable. 
> 
> IOW, the benchmark comes down to the performance of the actor.
> 
> Under this setup, you could conceivably replace the FA also and have two
> actors interact with each other based on scripts. That might be great
> theater but it ain't a 'field study' unless you want to really stretch the
> meaning of that phrase.
> 
> I don't know if you actually watched FAs in action but good ones, like all
> good salespeople, are very attuned to verbal cues and body language. They
> have trained themselves over countless such interviews to parse for such
> cues and redirect the conversation accordingly. The actor's performance,
> whether on or off script, will certainly have material impact on that
> two-way conversation, at which point you're conducting a play not a field
> study.
> 
> I do architecture/interface/design autopsy and surgery for a living. When I
> look at projects where 'usability tests' were conducted, virtually every
> single one of them will have an extremely predictable source of failure: the
> inability to control variables in multivariate tests. Variables have
> cumulative and interdependent impact on each other and the whole. It's the
> easiest urge in the world to say, well, we don't have time, budget or the
> analytical insight to properly pair them.
> 
> In this case, you are letting 2 of the 3 variables get corrupted by relying
> on the performance of an actor. That's not analytically sound testing
> practice and your data is bound to be suspect.
> 
> If you want to get a general idea of what FAs do, none of this matters. But
> if you want to unearth deeper and specific truths, you're in make-believe
> land.