[Asis-l] CFP: SIGIR 2011 Workshop on Crowdsourcing for Information Retrieval

[Matt Lease]

Crowdsourcing for Information Retrieval: A SIGIR 2011 Workshop
https://sites.google.com/site/cir2011ws/

CALL FOR PAPERS

The advent of crowdsourcing is driving a disruptive shift in IR areas
such as evaluation, learning to rank, and development of new hybrid
man+machine systems which blend automation and crowd computation
(potentially in real-time) to deliver innovative functionality and
search experiences. Traditionally manual-labor intensive tasks like
judging relevance and annotating training data can now be accomplished
more quickly and accurately, and at a fraction of traditional
costs.Despite this potential and early successes of crowdsourcing on
IR, many significant challenges remain with regard to theory,
methodology, policy, and best practices that currently limit our
ability to realize this potential in practice.

We invite submissions of papers describing novel, unpublished research
addressing one or more of the following areas:

* General: Theoretical, experimental, and/or methodological
  developments advancing state-of-the-art knowledge of crowdsourcing for
  IR

* Applications: search blending automation with the crowd, especially
  real-time systems which must model dynamic and temporal properties of
  crowd behavior

* New functionality: use of crowdsourcing to realize innovative
  search features (e.g. using geographic dispersion of the crowd for
  local search or to detect geo-specific intents, etc.)

* Machine learning: consensus labeling for vote aggregation, active
  learning strategies for efficient labeling, learning to rank with
  noisy crowd labels and multi-labeling

* Evaluation: evaluating systems with noisy and multi-labeled
  relevance judgments

* Infrastructure: new software packages and tool kits which simplify
  or otherwise improve general support for crowdsourcing or particular
  tasks (e.g. TurkIt, Get Another Label)

* Human factors and task design: how to design effective interfaces
  and interaction mechanisms for the crowd; how to enable effective
  crowd performance on tasks traditionally requiring scare and expensive
  domain experts; how different forms of crowdsourcing or crowd
  motivations (fun, socialization, prestige, economic, etc.), might be
  selected or tailored for different IR tasks (e.g. Page Hunt)

* Vision: Reflective or forward-looking position papers on use of
  crowdsourcing for IR

CROWDSOURCING CHALLENGE

An exciting and unique opportunity of this workshop will be significant
sponsorship by CrowdFlower, including both financial support and
access to technology, personnel, and guidance. Participation in the
Crowdsourcing Challenge is optional and distinct from the workshop's
Call for Papers.

Seed Funding: The first 20 researchers who commit to submitting a
paper (and presenting it at the workshop) will receive $100 in CrowdFlower
credits to design a prototype of their project.

Grant Funding: We anticipate making 2-4 CrowdFlower grants totaling $1500.
To request a CrowdFlower grant of up to $1000. The amount of funding awarded
will be determined on the basis of technical merit, and the authors
and titles of funded proposals will be published online.

See web site for more details: 
https://sites.google.com/site/cir2011ws/challenge


IMPORTANT DATES

  Workshop Papers
   Submissions: June 1, 2011
   Notification of acceptance: June 23, 2011
   Camera-ready: July 1, 2011

2011 CrowdSourcing Challenge:
   Grant proposals due: April 22, 2011
   Notification of funded grants: April 29, 2011

Workshop: July 28, 2011


SUBMISSION

Papers should be 8 pages or less, including all references and
figures. Papers must follow SIGIR 2011 formatting guidelines and be
submitted in either Adobe Portable Document Format (PDF) or Microsoft
Word Format (doc). Please ensure that any special fonts used are
included in the submitted documents. Papers should be submitted via
EasyChair (details forthcoming).


WORKSHOP ORGANIZERS

Vaughn Hester, CrowdFlower, USA
Matthew Lease, University of Texas at Austin, USA
Alex Sorokin, CrowdFlower, USA
Emine Yilmaz, Microsoft, UK

You can email the organizers at cir2011-org *at* googlegroups *dot* com.

PROGRAM COMMITTEE

Omar Alonso       Microsoft Bing
Paul Bennett      Microsoft Research
Adam Bradley   Amazon.com
Ben Carterette  University of Delaware
Charlie Clarke   University of Waterloo
Harry Halpin 	University of Edinburgh
Jaap Kamps        University of Amsterdam
Martha Larson   Delft University of Technology
Gabriella Kazai  Microsoft Research
Mounia Lalmas  University of Glasgow
Edith Law         Carnegie Mellon University
Don Metzler       University of Southern California
Stefano Mizzaro  University of Udine
Stefanie Nowak   Fraunhofer IDMT
Iadh Ounis           University of Glasgow
Mark Sanderson  RMIT University
Mark Smucker     University of Waterloo
Ian Soboroff         National Institute of Standards
Siddharth Suri    Yahoo! Research

-- 
Matt Lease
Assistant Professor
School of Information
University of Texas at Austin
http://www.ischool.utexas.edu/~ml