[Sigia-l] A matter of reliability

[Christopher Fahey]

> > do you believe and trust Ioannidis' study?
> I refer you to my sig. :-)

Ironically, your sig is the motto of the Royal Society, the first real
scientific "peer review" system. At the time, the only other peer review was
the Inquisition!! It really comes down to whether or not you trust the
peers.

Despite the sensational opening paragraphs, this article doesn't tell me
that peer-reviewed science is wrong in principle -- it seems to instead
suggest that the current system is largely and alarmingly not adhering to
its own principles. Bias, lazyness, nepotism, and fraud are widespread
contaminants in the methodology. Big surprise! 

The alternative, of course, to peer-reviewed science is not science at all,
but a world of ideas we accept based only on how intellectually or
emotionally compelling we as individuals find an idea -- without any weight
placed on what other people we might even recognize as "experts" think of an
idea. It's intellectual anarchy, and leads to things like alien abductions,
lucky rabbit's feet, dinosaurs in the Garden of Eden, and ouija boards. 

Still, I know there are limits to what can legitimately and/or usefully be
described or measured by current science. As far as IA/UX goes, I generally
still put more weight on experience, talent, and skill (i.e., 'expertise')
over supposedly more scientific approaches. For example, a great deal of the
value of most usability testing comes in the interpretation of the
non-statistical aspects of the results, in the professional/expert analysis
of the session.  

I wonder if the field of IA/UX would be better off if we all understood the
following equation:

   MT = a + b log2(2A/W + c)

Or is it enough that we understand the following, non-scientific and easily
understood phrasing of the same concept:

   "The time to acquire a target is a function of the
   distance to and size of the target."

Or even:

   "It's harder for users to click links that are 
   small and far away."

This is what JJG called, years ago, the "lab coats" aspect of IA. 

-Cf