[Sigia-l] ROI/Value of Search Engine Design - Resources?

[rich at richardwiggins.com]

Answers interleaved...

I certainly don't believe that the same ratio of searchers versus browsers
applies at all sites, or even within a category of sites.   I may believe,
for instance, that PC Connection's search engine is great, and the Amazon's
sucks (example only).  Once I know that, I'll tend to use the engine that
works more, and use other approaches at the site where search sucks.

> If the Search Dominance Theory (which says that some 
> percentage of users always uses Search) is true, then, while observing 
> users use different sites, we should see some percentage of those users 
> always using Search.

> Except we didn't. When we watched 30 users each use between three and six 
> sites, not a single one of them always used Search. 18% always used 
> categories, the rest were mixed usage -- some sites they used Search, some 
> sites they used categories.
 
Again, I'd expect variations among users, tasks those users had in mind, and
the sites they are trying to use.

Heck, I'd expect a huge variation just depending on whether the site has a
Search box, as opposed to a Search link, on the home page.

> Faster in what sense? When we time users, it isn't faster by clock time. 
> Search takes 15% to 20% longer. When we count clicks, the average on-site 
> Search takes 5.3 clicks, whereas, when using categories, the average is
4.8 
> clicks. Longer time and more clicks. (Sounds sorta like a beer commercial.)

I challenge you to a race.  You just overheard someone say "Thomas Friedman
had a really great piece in the Times on Peking Duct Tape."  I just went to
nytimes.com and found the piece in about 7 seconds. It took me two clicks.
You find it that fast by examining the zillion links on the Times' home page
and clicking, I'll buy you a beer.  :-)

> That's interesting. When I typed "Linksys WiFi" into Google, the first
page 
> that came up was this one: ....If I 
> understand you correctly, you're telling me that you'd click Search in the 
> upper left and wouldn't click on any of the Linksys WiFi product links. Is 
> that correct? If so, you'd definitely be in the minority of the users
we've 
> tested.

No no no, I might start with froogle.google.com to buy something, but I'd
never start a purchase at generic Google.  I was referring to local search. 
The comparison was that if PC Connection makes it easier to buy the same
known item at competitive prices due to superior search, then the one with
the better local search engine wins my business.

> It's interesting that, for your example, you chose the product genre of 
> computer accessories, because it's one of the worst performing genres 
> overall that we've measured. Almost all users go to Search in that genre 
> because the category links are usually dismal. (By the way, PC Connection 
> is one of the 21% of sites where every user in our study went to Search 
> immediately. I'm not surprised that you go straight to Search there 
> yourself. I know I do.)
 
I do use PC Connection's local search.  That's why I raised the example.  I
was comparing their local search with Amazon's. PC Connection's search is
superior for when I'm buying electronics.  QED!

> You won't hear any disagreement from me here. In fact, at the upcoming UIE 
> Research Forum (on Thursday of the User Interface 7 West conference: 
>we'll be discussing the results of a recent study 
> we've just completed that compares the search results from 76 different 
> on-site search engines in 7 different content genres. It's fascinating to 
> see the range of experiences that these sites deliver to users and what we 
> can learn by comparing them side-by-side.
> 
> That being the case, the question always is one of resource management. If 
> your development resources are limited, is it easier to completely
retrofit 
> your site's Search capabilities or to redo the categories? My guess is
that 
> it's a little bit of both.

You don't have to completely retro-fit the search service.  You build a Best
Bets service.  At the upcoming IA Summit in Portland, I will present a paper
on how easy it is to do, and ask why it is that everyone doesn't do it.

> As someone recently mentioned (maybe you--I can't remember), most content 
> requests fall into a standard Zipf distribution. Given that, designers 
> should take the 1%-10% of most sought after content and make sure the 
> category structure is flawless for delivering that content to the users. 
> Then you focus on the Search for the remaining 90% of the content that
less 
> than 10% of the users will request.

Yes it was I who mentioned Zipf. Your conclusion is diametrically opposed to
my own.  You take the top portion of the Zipf curve and you try to get it
right BOTH in your links and in your search.  You add a Best Bets service to
your search to handle the first 500-1000 unique search phrases, and you
leave the restto your purely robotic search engine, which is hopefully named
Google. 

I don't accept that it's more important to get the links right, and then
maybe you should worry about search functionality.  You should do BOTH. 
EVERYONE should do BOTH.  

When the Sapphire worm hit, you could visit Microsoft.com, and there was a
nice big link to a new page on Sapphire on the home page -- and there were
also relevant Best Bets for those who leapt to the search box.

We can quibble over what percent of the "n" saw the link, and what percent
went to the search box, but 100% were served.

It is a totally false dichotomy to act as if category/link construction and
search are separate activities -- and first you should worry about the
categories, and then, if you have time and resources left over, you should
try to patch search.   You can do both.  They interact.  The same Zipf curve
should inform both processes. 

/rich

_____________________________________________________

Richard Wiggins
Writing, Speaking, and Consulting on the Internet
rich at richardwiggins.com  http://richardwiggins.com