[Asis-l] Call for Demos: CIKM Living Labs for IR Evaluation Workshop (deadline: July 15)

[Krisztian Balog]

Workshop on Living Labs for Information Retrieval Evaluation @ CIKM 2013
CALL FOR DEMOS 

The idea of living labs, i.e., performing in-situ evaluation with real users, has been around for some time, but practical manifestations in an academic setting for IR evaluation are still in their infancy.  We invite researchers and practitioners to present their innovative prototypes or practical developments in a dedicated demo track at the CIKM Living Labs for Information Retrieval Evaluation (LL'13) workshop. Demo submissions must be based on an implemented system that pursues one or more aspects relevant to the <a href="http://ll2013.dcu.ie/cfp.html#topics">interest areas of the workshop</a>.  

Authors are strongly encouraged to target scenarios that are rooted in real-world applications. One way to think about this is by considering the following: as a company operating a website/service/application, what methods could  allow various academic groups to experiment with specific components of this website/service/application? 
In particular, we seek prototypes that define specific component(s) in the context of some website/service/application, and allow for the testing and evaluation of alternative methods for that component. One example is search within a specific vertical (such as product or travel search engine), but we encourage authors to think outside the (search) box.

All accepted demos will be evaluated and considered for the Best Demo Award.


Submission guidelines:

Demonstration papers must be submitted electronically through the <a href="https://www.easychair.org/conferences/?conf=ll2013">
EasyChair system</a>, in PDF format and formatted using the ACM SIGS format. 
(LaTeX and Word templates are available at http://www.acm.org/sigs/publications/proceedings-templates; for LaTeX, use the "Option 2" style.) 
Each submission should either include a URL for the web site of the system, or a URL of a shared code repository, or a URL to a 2-3 minute video demonstrating the system. 
The review process for demonstration submission is *not* double blind.
Accepted demo papers will be included in the CIKM 2013 Living Labs for Information Retrieval Evaluation Workshop proceedings, which will be indexed in the ACM Digital Library.


Important dates:

July 15, 2013: Deadline for demo submission (midnight Pacific Daylight Time)
July 22, 2013: Notification to authors
August 11, 2013: Camera-ready paper due
November 1, 2013: Workshop


Further Information:

Further information is available on the workshop website at http://LL2013.dcu.ie/ or by emailing the workshop organisers.


Workshop Organisers:

Krisztian Balog - University of Stavanger, Norway (krisztian.balog (at) uis.no)
David Elsweiler - University of Regensburg, Germany (david (at) elsweiler.co.uk)
Evangelos Kanoulas - Google, Switzerland (ekanoulas (at) gmail.com)
Liadh Kelly - Dublin City University, Ireland (liadh.kelly (at) computing.dcu.ie)
Mark Smucker - University of Waterloo, Canada (msmucker (at) uwaterloo.ca)