Robertson, S (Robertson, Stephen) On the history of evaluation in IR JOURNAL OF INFORMATION SCIENCE, 34 (4): 439-456 AUG 2008

[Eugene Garfield]

E-mail Address: ser at microsoft.com 

Author(s): Robertson, S (Robertson, Stephen) 

Title: On the history of evaluation in IR 

Source: JOURNAL OF INFORMATION SCIENCE, 34 (4): 439-456 AUG 2008 

Language: English 

Document Type: Article 

Author Keywords: cranfield; evaluation; experimentation; information 
retrieval; relevance; TREC 

Keywords Plus: INFORMATION-RETRIEVAL; RELEVANCE; ASK 


Abstract: This paper is a personal take on the history of evaluation 
experiments in information retrieval. It describes some of the early 
experiments that were formative in our understanding, an goes on to 
discuss the current dominance of TREC ( the Text REtrieval Conference) and 
to assess its impact. 

Addresses: Microsoft Res, Cambridge CB3 0FB, England 

Reprint Address: Robertson, S, Microsoft Res, 7JJ Thomson Ave, Cambridge 
CB3 0FB, England. 

E-mail Address: ser at microsoft.com 

Cited Reference Count: 36 

Times Cited: 0 

Publisher: SAGE PUBLICATIONS LTD 

Publisher Address: 1 OLIVERS YARD, 55 CITY ROAD, LONDON EC1Y 1SP, ENGLAND 

ISSN: 0165-5515 

DOI: 10.1177/0165551507086989 

29-char Source Abbrev.: J INFORM SCI 

ISO Source Abbrev.: J. Inf. Sci. 

Source Item Page Count: 18 

Subject Category: Computer Science, Information Systems; Information 
Science & Library Science 

ISI Document Delivery No.: 334XN 

INFORM PROCESSING MA 39 : 153 2003 

*NAT AC SCI
P INT C SCI INF WASH : 1958 

*NIST
TEXT RETR C : 

*ROYAL SOC
ROYAL SOC SCI UNPUB : 1948 

BELKIN NJ
ANOMALOUS STATES OF KNOWLEDGE AS A BASIS FOR INFORMATION-RETRIEVAL 
CANADIAN JOURNAL OF INFORMATION SCIENCE-REVUE CANADIENNE DES SCIENCES DE L 
INFORMATION 5 : 133 1980 

BELKIN NJ
ASK FOR INFORMATION-RETRIEVAL .1. BACKGROUND AND THEORY 
JOURNAL OF DOCUMENTATION 38 : 61 1982 

BELKIN NJ
ASK FOR INFORMATION-RETRIEVAL .2. RESULTS OF A DESIGN STUDY 
JOURNAL OF DOCUMENTATION 38 : 145 1982 

BLAIR DC
Some thoughts on the reported results of TREC 
INFORMATION PROCESSING & MANAGEMENT 38 : 445 2002 

CLEVERDON CW
FACTORS DETERMINING : 1966 

CLEVERDON CW
REPORT TESTING ANAL : 1962 

CROFT WB
ADV INFORM RETRIEVAL : 2000 

CROFT WB
LANGUAGE MODELLING I : 2003 

HANCOCKBEAULIEU M
J DOC 53 : 8 1997 

JONES KS
AUTOMATIC KEYWORD CL : 1971 

JONES KS
INFORM PROCESS MANAG 36 : 779 2000 

JONES KS
INFORM RETRIEVAL EXP : 1981 

JONES KS
J DOC 28 : 11 1972 

JONES KS
REPORT DESIGN STUDY : 1977 

JONES KS
REPORT NEED PROVISIO : 1975 

KEEN EM
5454 BLRDD 1978 

KEEN EM
ABERYSTWYTH INDEX LANGUAGES TEST 
JOURNAL OF DOCUMENTATION 29 : 1 1973 

KEEN EM
SMART RETRIEVAL SYST : 74 1971 

LANCASTER FW
MEDLARS - REPORT ON EVALUATION OF ITS OPERATING EFFICIENCY 
AMERICAN DOCUMENTATION 20 : 119 1969 

MARON ME
ON RELEVANCE, PROBABILISTIC INDEXING AND INFORMATION RETRIEVAL 
JOURNAL OF THE ACM 7 : 216 1960 

MITEV N
39 BRIT LIB 1985 

ODDY RN
INFORMATION-RETRIEVAL THROUGH MAN-MACHINE DIALOG 
JOURNAL OF DOCUMENTATION 33 : 1 1977 

ROBERTSON S
Understanding inverse document frequency: on theoretical arguments for IDF 
JOURNAL OF DOCUMENTATION 60 : 503 DOI 10.1108/00220410560582 2004 

ROBERTSON SE
ON THE EVALUATION OF IR SYSTEMS 
INFORMATION PROCESSING & MANAGEMENT 28 : 457 1992 

ROBERTSON SE
RELEVANCE WEIGHTING OF SEARCH TERMS 
JOURNAL OF THE AMERICAN SOCIETY FOR INFORMATION SCIENCE 27 : 129 1976 

ROBERTSON SE
Overview of the Okapi projects 
JOURNAL OF DOCUMENTATION 53 : 3 1997 

ROCCHIO JJ
SMART RETRIEVAL SYST : 313 1971 

SALTON G
SMART RETRIEVAL SYST : 1971 

SWANSON DR
SOME UNEXPLAINED ASPECTS OF CRANFIELD TESTS OF INDEXING PERFORMANCE 
FACTORS 
LIBRARY QUARTERLY 41 : 223 1971 

TURTLE H
EVALUATION OF AN INFERENCE NETWORK-BASED RETRIEVAL MODEL 
ACM TRANSACTIONS ON INFORMATION SYSTEMS 9 : 187 1991 

VOORHEES EM
TREC EXPT EVALUATION : 2005 

WALKER S
ONLINE CATALOGUE DEV : 84 1989