[Sigdl-l] EXTENDED DEADLINE: Call for Papers SAUCE 2016 - 1st Int Workshop on System And User Centered Evaluation Approaches in Interactive Information Retrieval

[Nicola Ferro]

EXTENDED DEADLINE January 22, 2016: Call for Papers: 1st International Workshop on System And User Centered Evaluation Approaches in Interactive Information Retrieval (SAUCE 2016)
co-located with 1st ACM SIGIR Conference on Human Information Interaction and Retrieval (CHIIR 2006)
17 March 2016, Chapel Hill, North Carolina, USA
http://sauce2016.dei.unipd.it


Nowadays, IIR continues to increase in complexity: user tasks and needs are demanding; data and information systems are rapidly evolving and greatly heterogeneous; and the interaction between users and IR systems is much more articulated. For example, consider what Web search is today: highly diversified results are returned from Web pages, news, social media, image and video search, products and more, and all are merged through adaptive strategies driven by current and previous user-systems interactions. As a result, experimental evaluation needs to appropriately model these evolving tasks, needs, data sources and user interactions. An additional challenge pertains to the anticipated outcome of IIR research and application. It is no longer sufficient to focus solely on precision, recall and satisfaction: successful IIR systems must engage, inform, and relate to users, taking into account single session and more long-term use and re-use.

To progress current evaluation methodologies and ensure they are able to effectively support the development of next generation IIR systems, one of the most compelling prospects is to bridge system-oriented and user-oriented evaluation methods. Both methodological approaches have their advantages and drawbacks: while system-centered methods ensure greater internal validity, they may fail to take into account user and contextual factors that influence IIR; user-oriented methods may better approximate actual user behavior, affect and cognition, but provide less experimental control of independent variables.

The goal of this workshop is to unite system- and user-centered IIR researchers for the purposes of:

        • Sharing different user-centered and system-centred research methods, measures, and tools in order to foster knowledge exchange;
        • Exploring the addition of user-centered evaluation strategies to system-oriented studies, and vice versa; and
        • Initiating collaborations between user- and system-oriented researchers to further IIR research.

The workshop will include a keynote with Dr. Eric Wiebe, North Carolina State University, and expert panel with Drs. Jaap Kamps, Diane Kelly, and Jacek Gwizdka.

Important Dates
Submission deadline: January 22, 2016
Notification of acceptance: February 12, 2016
Camera ready: February 26, 2016
Workshop day: March 17, 2016
Conference days: March 13-17, 2016

Call for Position Papers
General areas of interests include, but are not limited to, the following topics:

        • User-centred approaches that incorporate elements of system-centred evaluation
        • System-centred approaches that incorporate elements of user-centred evaluation
        • Phenomenon of interest or problems in interactive information retrieval that could be best approached using a combination of user- and system-centred evaluation
        • Descriptions of research or system design that are incorporate system- and user-centred methods
        • Correlation between system-centred and user-centred evaluation
        • Prediction of user behaviour, satisfaction, engagement, ... from system-centred evaluation

Papers should be formatted according to the ACM SIG Proceedings Template.
Papers should be two-four pages (maximum) in length.
Papers will be peer-reviewed by members of the program committee through double-blind peer review, i.e. authors must be anonymized. Selection will be based on originality, clarity, and technical quality. Papers should be submitted in PDF format to the following address:
https://easychair.org/conferences/?conf=sauce2016
Accepted papers will be published online as a volume of the CEUR-WS proceeding series.

Organizers
Heather L. O'Brien, University of British Columbia, Canada,  h.obrien at ubc.ca
Nicola Ferro, University of Padua, Italy ferro at dei.unipd.it
Hideo Joho, University of Tsukuba, Japan hideo at slis.tsukuba.ac.jp
Dirk Lewandowski, Hamburg University of Applied Sciences, Germany dirk.lewandowski at haw-hamburg.de
Paul Thomas, Microsoft, Australia, paul.thomas at csiro.au
Keith van Rijsbergen, University of Glasgow, UK, cornelis.vanrijsbergen at glasgow.ac.uk