[Sigdl-l] cfp CENTRE at CLEF

[Maria Maistro]

*Call for participation:CLEF, NTCIR, TREC Reproducibility, CENTRE at CLEF 2018*















*Lab held during the Conference and Labs of the Evaluation Forum, CLEF
2018September 10-14, 2018, Avignon,
Francehttp://www.centre-eval.org/clef2018/index.html
<http://www.centre-eval.org/clef2018/index.html>AIMS AND
SCOPE--------------------------The goal of CENTRE at CLEF 2018 is to run a
joint CLEF/NTCIR/TREC task on challenging participants to reproduce best
results of the most interesting systems submitted in previous editions of
CLEF/NTCIR/TREC and to contribute back to the community the additional
components and resources developed to reproduce the results (lab flyer
available at this link:
http://www.centre-eval.org/clef2018/centre_clef2018_flyer.pdf
<http://www.centre-eval.org/clef2018/centre_clef2018_flyer.pdf>).The
CENTRE at CLEF 2018 lab will offer two pilot tasks:Task 1 - Replicability: the
task will focus on the replicability of selected methods on the same
experimental collections;Task 2 - Reproducibility: the task will focus on
the reproducibility of selected methods on the different experimental
collections.Since CENTRE is a joint CLEF/NTCIR/TREC activity, participants
in Tasks 1 and 2 will be challenged to reproduce methods and systems
developed in all the evaluation campaigns, i.e. CLEF will challenge against
NTCIR and TREC results in addition to CLEF results.To participate in the
lab, the groups need to register by April, 27 at the following link:
http://clef2018-labs-registration.dei.unipd.it/
<http://clef2018-labs-registration.dei.unipd.it/>PAPERS TO BE
REPRODUCED--------------------------------------------Among the systems
submitted to the CLEF/NTCIR/TREC ad-hoc tasks we selected the following
papers:* Guyot, J., Radhouani, S., and Falquet, G. (2005). Ontology-Based
Multilingual Information Retrieval. In Working Notes for the CLEF 2005
Workshop.* Nguyen, D., Overwijk, A., Hauff, C., Trieschnigg, D. R.,
Hiemstra, D., and De Jong, F. (2008). WikiTranslate: Query Translation for
Cross-lingual Information Retrieval Using only Wikipedia. In Working Notes
for the CLEF 2008 Workshop.* Sorg, P., and Cimiano, P. (2008).
Cross-lingual Information Retrieval with Explicit Semantic Analysis. In
Working Notes for the CLEF 2008 Workshop.* Yang, P., and Fang, H. (2013).
Evaluating the Effectiveness of Axiomatic Approaches in Web Track. In TREC
2013.* McCreadie, R., Deveaud, R., Albakour, M., Mackie, S., Limsopatham,
N., Macdonald, C., Ounis, I., and Thonet, T. (2014). University of Glasgow
at TREC 2014: Experiments with Terrier in Contextual Suggestion, Temporal
Summarisation and Web Tracks. In TREC 2014.* Gallagher, L., Mackenzie, J.,
Benham, R., Chen, R. C., Scholer, F., and Culpepper, J. S. (2017). RMIT at
the NTCIR-13 We Want Web Task. Proceedings of
NTCIR-13.GUIDELINES------------------Groups should satisfy the following
guidelines to participate in this lab:* They need to use an open source IR
system, e.g. Lucene, Terrier, Indri, Atire;* The runs should be submitted
in TREC format;* The code used to produce the runs should be uploaded in a
Bitbucket repository provided by the organizers upon the registration to
CENTRE at CLEF.IMPORTANT DATES-----------------------------* Registration
closes: April 27, 2018* Runs due from participants: May 11, 2018*
Submission of participant papers: May 31, 2018* Notification of acceptance:
June 15, 2018* Camera ready due: June 29, 2018* CLEF 2018 conference:
September 10-14, 2018ORGANIZERS--------------------* Nicola Ferro,
University of Padua, Italy* Maria Maistro, University of Padua, Italy*
Tetsuya Sakai, Waseda University, Japan* Ian Soboroff, NIST, US*
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.asis.org/pipermail/sigdl-l/attachments/20180329/a8c49f4b/attachment.html>