McDaniel RE: [Sigia-l] ROI/Value of Search Engine Design - Resources?

[Richard Hill]

[Re-posted because of html coding.  Dick Hill

At 11:21 AM 2/19/2003 -0600, Mike.Steckel at SEMATECH.Org wrote:
>I am sceptical of this on my first read. How is it determined that Albert 
>is a casual searcher in the system? How is he "assigned" as a casual 
>searcher? Also, following the berrypicking model 
>(<http://www.gseis.ucla.edu/faculty/bates/berrypicking.html>http://www.gseis.ucla.edu/faculty/bates/berrypicking.html) 
>there is a good chance that Albert would increasingly move toward being an 
>Interested Layperson during the same search session, especially as he 
>narrows the search to something more specific, and then possibly back to 
>Casual Searcher again as he talks to the person requesting a book. All 
>during the same search. I just don't see the real world falling into these 
>neatly defined categories. I also, don't see people being able to 
>effectively assign themselves to the categories.

I come from a usability and UI design area, so my answer to how to classify 
Albert a casual searcher would be to do enough user analysis to be able 
make that decision.  I don't trust people to be able to classify themselves 
-- I'd rather do it through a combination of interview questions and 
observation.  Since the Type of User category reflects primarily how much 
knowledge a person already has about the subject of their search, we'd have 
to engage in a discussion with users to see what their level of knowledge 
is.  It also includes an element of motivation -- how likely is Albert to 
conclude that there's nothing there and report back to the customer that 
they don't have anything?  That may in fact be two separate categories, but 
they seem to lead to similar heuristics.

People can certainly change levels within a category, but as presented 
below it wouldn't happen in the same search session.  My understanding of 
the berrypicking article was that the act of searching changes not only the 
query but the information need itself.  That's different than a person's 
level of knowledge about their subject.  (Nice link, by the way -- thanks.)

Where I see the berrypicking fitting in better is as a source for 
heuristics in the Search Experience category.  Novice searchers and many 
intermediate searchers don't think in terms of entire, multi-query search 
strategies.  If you only do one, maybe two queries at the most, it's hard 
for the berrypicking effect to kick in.  Perhaps a better way to say it 
would be that the more queries your overall search takes, the more 
berrypicking will have an effect on the final results set.  A UI should 
allow intermediate and experienced searchers to take advantage of the 
serendipitous findings on the way to their initial target.

As for the real world fitting neatly into these categories -- the function 
of the model is to simplify the many parameters affecting search to the 
point that we can effectively make UI design decisions.  Are you skeptical 
of these specific categories or of the idea of having such categories in 
the first place?  How do you approach the various "it depends" questions 
that come up when designing specific user interfaces?

Thanks,

      Scott McD.

>-----Original Message-----
>From: Scott McDaniel [mailto:scottmcd at cognetics.com]
>Sent: Tuesday, February 18, 2003 4:08 PM
>To: sigia-l at asis.org
>Subject: RE: [Sigia-l] ROI/Value of Search Engine Design - Resources?
>I don't have an MLS, but my wife does.  We recently co-authored an article 
>that appeared in the second volume of UPA's User Experience magazine.  The 
>complete reference is:
>McDaniel, S. and McDaniel, M. (2002) The Big Dig: Mining Nuggets of Value. 
>User Experience, 1:2, 20-29.
>In the article we note that the current heuristics about how to produce a 
>good search interface were so general as to be next to useless.  Since the 
>answer to specific questions always seems to be "it depends," we proposed 
>three dimensions upon which "it" could depend.  Most of the heuristics 
>came from current literature (NNG's report on search, Jared's book, LIS 
>textbooks, Tog on Interface, etc).  Some are proposed heuristics without 
>research support.  If you'd like the complete set of heuristics, see the 
>article if you have access to the magazine or contact me off list.
>The basic idea is that you determine where your users fall on each of 
>these three dimensions:
>Type of User (How familiar is the user with the information being searched?)
>   -- Casual Searcher (have a passing interest in the material.  Most 
> e-commerce shoppers.)
>   -- Interested Layperson (understands basic jargon and wants to learn more)
>   -- Subject Matter Expert (thorough understanding of the material)
>Search Experience (How much experience or training has the user had in 
>searching?)
>   -- Novice Searcher (typically enter a search word and press enter)
>   -- Intermediate Searcher (understand basic Boolean searching)
>   -- Advanced Searcher (are trained search professionals)
>Goal of the Search (What type of information is the object of the search?)
>   -- Precision Search (locates a specific, known item)
>   -- Recall Search (locates all items that meet given criteria)
>   -- Some Good Items (locates a small set of items that best meet given 
> criteria)
>Each value on these dimensions has a small set of associated 
>heuristics.  By determining where your user falls on each scale, you 
>derive a full set of heuristics for the search interface.  I'm including a 
>brief excerpt from the article below.
>I've seen a couple of Jared's presentations (most recently the Scent of 
>Information, given to the UPA DC Metro chapter).  My impression is that 
>most of the people he tests would be:
>   Casual Searchers or Interested Laypersons
>   Novice Searchers
>   Precision or Some Good Items
>I'd hesitate to generalize findings for these people to all Search 
>situations.  (Jared -- Are my impressions above correct?)
>     Scott McD.
>
>
>Here's the specific excerpt I mentioned earlier:
>Let s take the case of Albert the Bookstore Associate to illustrate how 
>this works.
>Albert is receiving many requests for books about Alzheimer s.  Because 
>Albert is not an expert on Alzheimer s or particularly motivated beyond a 
>customer s request, he is best classified as an Casual Searcher, and 
>because he, like most bookstore employees, have limited search experience, 
>he is best classified as a Novice searcher.  Since his customers don t 
>have specific books in mind, Albert must perform a some good items search 
>to recommend a few choices.  Combining these results, leads to the 
>following set of guidelines:
>
>§       Offer a simple search box on the home page and on each page 
>throughout the site.
>§       Present information in the search results that allows users to 
>assess relevance.
>§       Allow Casual Searchers in an e-commerce setting to search for 
>things other than products.
>§       Allow Casual Searchers to search for items with their own vocabulary.
>§       Consider alternatives to a  search.
>§       Global search is better than a scoped search.
>§       Lead Novice searchers through the search dialogue.
>§       Present the search results clearly and simply.
>§       Make it easy to evaluate individual result items.
>§       Allow users to save and resume Recall searches. (Also applies to 
>some good items )
>§       Make it easy to expand searches as well as narrow them.
>§       Search globally.
>§       Accept word variants, word-stemming, and synonyms for search terms.
>§       Make a simplified reference interview an optional part of the 
>search process.
>§       Support the task of creating a short list of results.
>Inspecting the guidelines reveals some redundancies, such as Global search 
>is better than scoped search and Search globally.   These are easily 
>consolidated.  But what happens if the guidelines conflict?  Short of 
>flipping a coin, the best way to make a decision is to look at data from 
>your user analysis.  Give more weight to the needs of your primary 
>users.  You might also consider whether you in fact need two different 
>user interfaces (text entry search vs. a category browse), or even two 
>different products.
>
>
>
>
>
>
>~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>Scott McDaniel
>Cognetics Corporation, Designer
>1320 Fenwick Lane, Suite 209, Silver Spring, MD 20910
>301.587.7549   --  scottmcd at cognetics.com  --  fax 301.562.8267
>~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~



~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Scott McDaniel
Cognetics Corporation, Designer

1320 Fenwick Lane, Suite 209, Silver Spring, MD 20910
301.587.7549   --  scottmcd at cognetics.com  --  fax 301.562.8267
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Executive Director
American Society for Information Science and Technology
1320 Fenwick Lane, Suite 510
Silver Spring, MD  20910
FAX: (301) 495-0810
PHONE: (301) 495-0900

http://www.asis.org