[Sigia-l] Comparing search engines

[Lyle_Kantrovich at cargill.com]

Ziya,

Not sure that I understand exactly what you're asking.  You point out a 
lot of aspects of the "system":
- lots of data
- indexed or indexable
- routine DB methods
- queries
- etc.

If you're saying that the current system traits may have nothing to do 
with "best results" (or even "good" results), then I whole-heartedly 
agree.  The current system may never support good results - it may be 
the wrong system.

Lets talk about these instead:
- users
- their needs, goals and expectations
- what they know
- the kinds of searches people would want to do (e.g. known item vs. 
subject search)
- the kinds of data: structured/unstructured
- the likelihood that the "data" can meet their needs
- the likelihood that they can compose a query to find the data to meet 
their needs
- how they interface to the data i.e. search and retrieval

Just because you have lots of data does not mean the user can 
effectively access it via a search interface.

- Maybe you need a browse interface
- Maybe you need an index
- Maybe you need a site map
- Maybe you need less data
- Maybe you need a human (ooh, blasphemy! I'm goin' straight to IT hell 
for that one.)

Okay, forget what I just said.  I love hearing about what people think 
makes a good search engine good.  This is great discussion.  Just keep 
in mind that we're taking a big leap of faith when we assume that a 
search interface *can* be good for your specific users in context of 
their needs and your specific data.  

Here's an example of why "good" results are hard to measure:

Imagine you search the web for "site map" what would you expect to get?  


Now ask yourself: Who are you?  What do you know about site maps?  What 
do you want to see or know about site maps?  Do you want an example?  Do 
you want to know how to make one, why to make or how they are used?  Do 
you want to find a tool for making and maintaining one?  Do you want a 
site map of the search engine site you typed it into?  Do you want a 
site map of the Internet?  Do you want to know how to create a 
geographic site map from aerial photos of your agricultural land?

Regards,

Lyle

----
Lyle Kantrovich
User Experience Architect

Croc O' Lyle - Personal Commentary on usability, information 
architecture and design.
http://crocolyle.blogspot.com/

"Simplicity is the ultimate sophistication." 
	- Leonardo da Vinci
 

-----Original Message-----
From: listera at rcn.com [mailto:listera at rcn.com]
Sent: Thursday, June 19, 2003 11:50 AM
To: sigia-l at asis.org
Subject: Re: [Sigia-l] Comparing search engines


"donna at maadmob.net" wrote:

> How do you figure out what the 'best results' are to compare???

You mention that everything is in "a big store of data". I take it to 
mean
that the data to be searched is already indexed (or indexable) via 
routine
database retrieval methods. If that's the case, your search engine is 
really
a front-end for query construction, to a predictable database. Result 
sets
returned should be predictable and non-variable.  In this context, 
what's
'best results'? 

Ziya
Nullius in Verba