[Sigia-l] re: the future of search

[Listera]

"Michael Fry" wrote:

>> Would you want a librarian who may *not* have heard about urinating rats or
>> flying mexicans to help you or, say, a search engine that can parse 5 billion
>> docs a la Google to get you into the ballpark in less than a second?
> 
> One of the myths surrounding librarians is that their value derives from
> what they *know*...

(Since much of this has been covered in a previous thread "Insight moves
sight to the site" I had with James Weinheimer I won't repeat all that,
but...)

The reason I excluded the possibility of the 'knowing' librarian was to even
out the playing field. I assume that in a library there are patterns of
often-repeated questions the answers to which are readily available from
librarians. With age, experience and familiarity with the library clientele,
a librarian amasses a goodly amount of such answers. (Sort of like the
cached memory of a search engine :-)

So let's look at what's really important here: trends.

1)  Less than ten years ago, if one wanted to engage in any semi-serious
research, one had to go to a library and, often, consult a librarian. Today,
hundreds of millions of people armed with nothing more than a PC and a
browser can get at an unbelievably rich source of material from around the
world, via search engines and other methods of referencing. This is a
colossal change. I've lived within walking distance of an Ivy-league library
most of my live. I know today I walk into a library far, far less frequently
than, say, ten years ago.

2)  Less than ten years ago, the general public had practically no
electronic search access. Then we had fairly primitive directories and
search engines in the first wave of the web. Those tools progressively got
better. Google upped the ante quite a bit. We are now seeing even more
sophisticated tools emerge. Again this is happening at an incredible rate.

3)  One of the myths surrounding search engines is that the first result you
get in a fraction of a second is the last one you look at. Not unlike the
librarian's interview with a client, working with a search engine is an
iterative process, too. In the five minutes or so it takes a librarian to
narrow down the search criteria, one could easily go through a similar
process of pruning a dozen or more times with a search engine. The search
engine may not be as intuitive as the librarian but it covers more in less
time and can lead you to your answer in progressively narrowing steps.

4)  The number of libraries/librarians (let alone the good ones) are
miniscule compared to the number of PCs around the world. Just as my
librarian consulting patterns have changed drastically in less ten years, a
generation or two into the future, public librarians may become endangered
species. (Though I see a much brighter future in the .biz/.edu world.)

5)  There's a very strong trend to better categorize data and information,
through XML, etc. This will take, obviously, a long time to accomplish. But
newly generated, electronic data may become nearly self-describing and far
more granular in the very near future. Our ability to pinpoint specific
information will thus be greatly enhanced.

6)   Over time, peoples' ability to efficiently use search engines and other
electronic referencing tools will improve, as will the UIs for such tools.
I'm not suggesting that everyone will become Boolean cowboys in a few years,
but let's not underestimate the progress made and new advances waiting in
the wings (from natural language parsing to speech recognition).

7)  Things change. It used to be that you could go to a small haberdasher
downtown, talk directly to the proprietor and benefit from his 27 years in
the business. Today, if you are in a Macy's in a mall and go to the hat
section you'd be lucky if the 19 year old girl working the counter can give
you the correct day of the week. So a generation brought up on search
engines may not even appreciate the librarians' contribution.

Now I'm not saying librarians are becoming quaint. It's the economics that's
changing the equation. The distributed 'efficiency' of hundreds of millions
of search engines sitting on peoples' PCs is hard to compete against. Their
answers may not as precise as a librarian's but they may be just good
enough. Sad to say but good enough in volume pretty much trumps everything
else in life.

Best,

Ziya