[Asis-l] INEX 2011: Call for Participation

[Jaap Kamps]

INEX 2011
Call for Participation

REGISTRATION NOW OPEN AT http://inex.mmci.uni-saarland.de/

OVERVIEW

Traditional IR focused on pure text retrieval over "bags of words" but 
the use of structure---such as document structure, semantic metadata, 
entities, or genre/topical structure---is of increasing importance on 
the Web and in professional search.  INEX has been pioneering structured 
retrieval since 2002, by providing large test collections of structured 
documents, uniform evaluation measures, and a forum for organizations to 
compare their results.  INEX 2011 will proudly continue pushing research 
boundaries, with a wide range of tasks including Social Search, Faceted 
Search, and Snippet Retrieval.

TASKS AND TRACKS

- Books and Social Search Track

   Investigating techniques to support users in searching and navigating 
books, metadata and complementary social media.  The _Social Search for 
Best Books Task_ studies the relative value of authoritative metadata 
and user-generated content using a collection based on data from Amazon 
and LibraryThing.  The _Prove It Task_ asks for pages confirming or 
refuting a factual statement, using a corpus of the full texts of 50k 
digitized books.

- Data Centric Track

   Investigating retrieval over a strongly structured collection of 
documents based on IMDb.  The _Ad Hoc Search Task_ has informational 
requests to be answered by the entities in IMDb (movies, actors, 
directors, etc.).  The _Faceted Search Task_ asks for a restricted list 
of facets and facet-values that will optimally guide the searcher toward 
relevant information.

- Interactive Track

   Investigating the behavior of users when interacting with structured 
documents, using a collection based on Amazon book records and 
additional social data from LibraryThing, as well as developing access 
tools which are effective in user-based environments.

- Question Answering Track

   Investigating real-world focused information needs formulated as 
natural language questions using a Wikipedia corpus with additional 
annotation (document structure, sentences, and named-entities), 
evaluated by both a readable summary of question context and a list of 
answers.

- Relevance Feedback Track

   Investigate the utility of incremental passage level relevance 
feedback by simulating a searcher's interaction.  An unconventional 
evaluation track where submissions are executable computer programs 
rather than search results.

- Snippet Retrieval Track

   Investigate how to generate informative snippets for search results. 
  Such snippets should provide sufficient information to allow the user 
to determine the relevance of each document, without needing to view the 
document itself.

- Web Service Discovery

   Investigate techniques for discovery of Web services based on 
searching service descriptions provided in WSDL. The answer may be a 
single service that matches the requirements of a request, or a group of 
services that combined together match the requirements.

A detailed description of each of the tracks can be found at 
http://inex.mmci.uni-saarland.de/

IMPORTANT DATES

Schedules vary per track, with collections and topics coming available 
in the spring (now!), submissions due in the summer, and results coming 
available in the fall.  INEX 2011 concludes with a workshop in December:

12-14 Dec 2011:  INEX Workshop near Saarbruecken

RELEVANCE ASSESSMENTS

Some of the tracks rely on participating groups to build the test 
collection though topic creation and relevance assessments.  The Books 
and Social Search Track uses crowdsourcing.

WORKSHOP AND PROCEEDINGS

Participants will be able to present their approaches and the final 
results at the INEX workshop to be held in December near Saarbruecken. 
All participants will be able to publish their work in the workshop 
pre-proceedings.  Fully peer-reviewed and revised papers will be 
published the INEX post-workshop final proceedings.  Since 2004 the 
final proceedings have been published in the Springer's Lecture Notes in 
Computer Science (LNCS) series, and we seek to continue that tradition.