[Sigia-l] Dedicated to those...

[Steven Pautz]

It seems to me that Prady is focusing on the wrong aspects of this
topic: it doesn't matter whether or not measurement improves or hurts
design (or whatever else is being measured) -- the point is that
measurement changes the thing being measured.

The original link described a scenario in which programmer behavior
was altered in order to maximize performance along a particular
metric. The key insight here, I believe, is that the exact same thing
happens in design -- just with niche, function, and form in place of
programmer behavior.

The original conceptualization and formulation of design is based on
many abstract factors -- user and business needs and goals inspire and
define purpose, audience, and context, which guide and inform design
decisions (or whatever other model/methods you use). As Prady pointed
out, metrics are not a substitute for this; they only measure the
results as expressed in something measurable.

However, whatever measurements are made will invariably have an effect
on the design, and these effects will be unavoidably colored by the
implicit goal of improving measurement scores. Even if the
measurements are accurate, relevant, and appropriate, basing design
decisions on measurements instead of the aforementioned abstract
factors changes the overall context surrounding the design. In a
sense, it changes the design decisions from "purpose-centered" or
"user-centered" (or whatever else it's intended to be) to
"measurement-centered".

Even if the measurements are flawless, they should only be used to
inform/revise the abstract factors which drive design decisions --
using them to guide the decisions themselves changes the very nature
of design.

The actual companies considered in this debate are irrelevant. The
companies defined "success" for themselves based on their use of
measurements -- whether it be maximizing profits, minimizing risk,
minimizing environmental impact, or teaching children to read.
Observers whose definitions of "success" align with those measurements
will view the company as a success; others will view the company as a
failure. (After all, isn't personal judgment of success just another
measurement?)

Because of their measurement-driven pursuit of "success", as opposed
to a context-dependent "abstract factor"-based pursuit, many companies
have a difficult time achieving "success" in contexts outside of their
established scope.

Measurement changes the thing being measured, whether it be interface,
product, service, strategy, or whatever. Such changes alter the nature
of the design behind the interface/product/service/strategy, harming
its ability to suit -- or even perceive -- different purposes,
audiences, or contexts.

~Steven Pautz


On 11/19/06, prady <pradyotrai at gmail.com> wrote:
> Ziya Oz :
> > I have nothing against MBAs as a phylum, just the ones who think that
> > metrics (alone) can be a substitute for design as business strategy.
>
> Metrics is not substitute of what you do, it is a "measure" of whatever you do.
>
> I see a confusion about the term metrics/ROI. Let's simplify and call
> it "measure". And let's start asking yourself, is it stupid to believe
> in "measuring"? Ofcourse with the exception (Joel's case in point)
> when you "measure" with wrong yardstick.
>
> There's another confusion here that WalMart, BestBuy, Dell failed,
> because they couldn't get on the Music Store/design business (This is
> happening due to misunderstanding the term "business strategy"). You
> are measuring their success with wrong ward stick. All the examples I
> gave are market leaders in their own industry (including WalMart, Dell
> and Bestbuy). And the point was that they are leader because they
> "measure" precicely what they do.
>
> Through my examples of leading firms I was pointing towards the fact
> that everything is "measurable" and if you don't measure you can't
> "manage". You yourself pointed how APPL measures it's success (and
> design). Other firms are measuring manufacturing, service, HR,
> business profitability, customer satisfaction, value of customer,
> brand equity, human capital, sales, design & development to make these
> individual activities more managiable. Why would anybody disagree with
> that?
>
> Now coming to the subject, which you insist on - "design". It can be
> measured and should always be. That is not to say that this approach
> puts extra emphasis on measurement and not on design itself. You can
> make a case that something (design in this case) has not reached its
> maturity at a certain instance and "measurement" should be less
> discretionary for final judgements. And people in white coat will
> understand that.
>
> I am not sure where this debate is headed, but I believe that you are
> fairly intelligent guy and you get it. You have the unique attitude
> towards learning - you start by denial. The thing which are causing
> you to dance on both sides are due to unclarity of terms in your own
> mind about "best practices", "ROI", "innovation", "strategy",
> "metrics", etc. For that you should just simplify them and drink
> kool-aid.
>
> Good debate. Thank you.
> ------------
> When replying, please *trim your post* as much as possible.
> *Plain text, please; NO Attachments
>
> Searchable Archive at http://www.info-arch.org/lists/sigia-l/
>
> Most presentations/papers and posters have been loaded to the IA Summit 06 website:
>
> http://iasummit.org/2006/conferencedescrip.htm
> http://iasummit.org/2006/posters.htm
>
>
>
> ________________________________________
> Sigia-l mailing list -- post to: Sigia-l at asis.org
> Changes to subscription: http://mail.asis.org/mailman/listinfo/sigia-l
>