[Asis-l] CFP CIKM 2011 Workshop on Data infrastructurEs for Supporting Information Retrieval Evaluation - Extended Deadline 9 July 2011

[Nicola Ferro]

** Apologies for cross postings **

--------------------------------------------------------------------------------------------------
Call for Papers - Extended Deadline July 9, 2011

Data infrastructurEs for Supporting Information Retrieval Evaluation (DESIRE 2011) Workshop

Co-located with CIKM 2011, the 20th ACM Conference on Information and Knowledge Management

Friday 28th October 2011, Glasgow, UK

http://www.promise-noe.eu/events/desire-2011/
--------------------------------------------------------------------------------------------------

AIM AND SCOPE

The Information Retrieval area has a strong and long tradition dating back to the 1960s in producing 
and processing scientific data resulting from the experimental evaluation of search algorithms and 
search systems. This attitude towards evaluation has led to fast and continuous progress in the 
evolution of information retrieval systems and search engines. 

However, in order to make these data test collections understandable and usable they must be endowed 
with some auxiliary information, i.e., provenance, quality, context, etc. Therefore, there is a need 
for metadata models able to describe the main characteristics of evaluation data. In addition, in 
order to make distributed data collections accessible, sharable, and interoperable, there is a need 
for advanced data infrastructures.

Nevertheless, the information retrieval area has barely explored and exploited the possibilities for 
managing, storing, and effectively accessing the scientific data produced during the evaluation 
studies by making use of the methods typical of the database and knowledge management areas.

It is thus time for these three communities – information retrieval, databases, and 
knowledge management – to join efforts, meet, and cooperate to address the problem of envisaging and 
designing useful infrastructures able to coherently manage pertinent data collections and sources 
of information, and so take concrete steps towards developing them.

Indeed, the information retrieval experts need to recognise this need, while the database and 
knowledge management experts need to understand the problem and work together to solve it by using 
the methods and techniques specific to information management.

Therefore, the workshop will gather together experts from these three areas,  to encourage them to 
recognise the urgency of addressing the problem in an integrated and coherent way, and to coordinate 
efforts towards drawing a roadmap and suggesting best practices for an effective solution of the 
problem.

The objectives of the workshop are: to gather interested experts from the three areas of interest 
to CIKM and discuss how to address the problem of envisaging and designing evaluation 
infrastructures able to store, manage, and make accessible the scientific data and knowledge of 
interest for advancing the evaluation of information retrieval and access tools.

The goal of the workshop is to understand how to make use of the expertise of the three scientific 
areas in a cooperative way to avoid the duplication of efforts which may occur when addressing the 
problem separately in each specific area and to trigger synergies and joint actions on this issue. 

The workshop is intended to be a single full day workshop, consisting of: invited talks and paper 
presentations during the morning; active discussion groups during the afternoon, engaging the 
participants in the topics which have emerged in the previous sessions in order to define a roadmap 
and best practices.

The main outcome will be to produce a roadmap and initial best practices to guide the development 
of the necessary evaluation infrastructures. 


TOPICS

- Conceptual and logical data models for representing IR evaluation scientific data
- Metadata formats for describing scientific data produced during IR evaluation
- Knowledge management for IR experimental evaluation
- Data quality, provenance, adaptability and reusability in the IR evaluation
- Data pre- and post-processing, metrics, and analyses in the IR evaluation
- Data exchange, integration, evolution and migration for IR evaluation infrastructures
- Workflow, Web services and Web Service Composition for IR evaluation infrastructures
- Information Extraction and Text Mining for linking scientific literature and experimental data
- Data citation
- Evaluation, Test collections, Crowdsourcing for IR evaluation
- Visualization of scientific data coming from experimental evaluation


IMPORTANT DATES

- Individual Workshop Papers Due: July 9, 2011 (extended)
- Notification of Acceptance: July 18, 2011
- Camera Ready: July 22, 2011


FORMAT

Authors are invited to submit electronically original papers, which have not been published and 
are not under consideration elsewhere, using the ACM template Option 2. including the ACM metadata 
(General Terms, Categories, Keywords, Copyright number etc), available at:

http://www.acm.org/sigs/publications/proceedings-templates

Two types of comunications or position papers are solicited:

- communications: 4 pages max.
- position papers: 6 pages max.

Papers will be peer-reviewed by at least 3 members of the program committee. Selection will be 
based on originality, clarity, and technical quality. 

Papers should be submitted in PDF format to the following address: 

https://www.easychair.org/conferences/?conf=desire2011

KEYNOTE SPEAKERS

- Norbert Fuhr, University of Duisburg-Essen, Germany.
- Maurizio Lenzerini, Sapienza University of Rome, Italy.

PROGRAM COMMITTEE

- Stefano Ceri, Politecnico di Milano, Italy
- Maarten de Rijke, University of Amsterdam, The Netherlands
- Donna Harman, National Institute of Standards and Technology (NIST), USA
- Noriko Kando, National Institute of Informatics (NII), Japan
- Henning Müller, University of Applied Sciences Western Switzerland
- Shila Ofek-Koifman, IBM Haifa, Israel
- Christos Papatheodorou, Ionian University, Greece
- Fausto Rabitti, ISTI-CNR, Italy
- Mark Sanderson, RMIT University, Australia
- Timos Sellis, National Technical University of Athens, Greece
- Gerhard Weikum, Max-Planck-Institut für Informatik, Germany

ORGANIZERS

- Maristella Agosti, University of Padua, Italy
- Nicola Ferro, University of Padua, Italy
- Costantino Thanos, ISTI-CNR, National Council of Research, Italy

CONTACT 

- Nicola Ferro, ferro at dei.unipd.It