[Asis-l] Workshop: Measuring Web Search Effectiveness: The User Perspective

[Richard Hill]

[Forwarded by request of Carolyn Watters.  Dick Hill

			WWW 2004 CONFERENCE WORKSHOP

New York, May 18, 2004				www2004.org

			CALL FOR PARTICIPATION

	MEASURING WEB SEARCH EFFECTIVENESS: THE USER PERSPECTIVE

The goal of this workshop is to bring the users of the Web into focus by
bringing together researchers, developers, and information managers to
discuss both how to measure Web search effectiveness from the user
perspective and how to use this information to build better Web systems.
We plan to open a conduit so that results from research make their way into
new Web systems at the same time as development problems are brought to the
attention of the research community.

This workshop focuses on the user experience in seeking information on the
Web. We will explore the effectiveness of retrieval tools for users seeking
information on the Web, where the data set is extremely large and
heterogeneous with respect to content, structure, and quality. The goal is
to reflect on the impact of the user experience on the design and
development of new Web systems and tools.


		Paper Submission deadline: 	March 26, 2004
		Acceptance Notification:      April 5, 2004

			Workshop Description

While we have learned a great deal about creating large document spaces and
accessing these spaces, we know relatively little about the users who deal
with a multi-billion-page Web. Further research is needed to address the
user issues related to effectiveness and quality of experience when
interacting with Web search engines. The recent intensity of work on
personalization and filtering for pre and post search processing highlight
the gulf that remains for users. The semantic Web initiative holds promise
of better categorization of content but does not address issues of
evaluating human interaction with Web search engines, including the
usability and effectiveness of search tools.

Metrics and methodologies have been developed by the Information Retrieval
and Web communities to assess the effectiveness of large homogeneous
retrieval systems. However, new measures are needed for Web retrieval that
go beyond recall and precision and that are more user-centric. In the
proposed WWW Workshop we address issues of evaluating the effectiveness of
retrieval tools for users seeking information on the Web, where the data set
is extremely large and heterogeneous with respect to content, structure, and
quality.

The proposed Workshop will be a multi-disciplinary forum of presentations
and discussions of theoretical foundations, evaluation measures,
methodologies, case studies and user study results. The multidisciplinary
nature of this emerging area allows us to welcome a wide range of
participant interests from both the research and development communities.
While it is anticipated that most participants would have background
knowledge in either evaluation of user experiences or in providing Web
services, the discussion is not intended to be at a highly technical level
but rather on a conceptual and highly multidisciplinary level.

			Anticipated Outcome

 In addition to the Workshop presentations and discussion, we anticipate
producing a separate WWW2004 workshop proceedings and potentially as special
topic issue for a relevant journal.

		Submissions and Evaluations

Participants will be asked to prepare a 1 page statement of interest and
relevant experience and contributors are requested to submit a 4-8 page
paper on a topic of interest to the discussion. Papers will be reviewed by
at least 2 reviewers for acceptance in the proceedings.

Areas of interest include, but are not limited to:
	User-based Web search engines effectiveness measures including
		relevance, utility and usefulness
	Usability of Web search tools
	Evaluation of Web search tools in information seeking problems
	Role of genre in Web search
	Profiles and personalization to enhance Web search
	Effect of task on information seeking behavior on the Web
	Multimedia effectiveness measures
	Individual differences in Web search


		Paper Submission deadline: 	March 26, 2004
		Acceptance Notification:      April 5, 2004

			Submissions by email to
		Dr. Carolyn Watters  watters at cs.dal.ca
		Dr Amanda Spink  aspink at sis.pitt.edu


			Workshop Facilitators

Carolyn Watters is a Professor of Computer Science at Dalhousie University,
Halifax, Canada and the co-director of the Web Information Filtering Lab.
She has published widely on topics in Information Retrieval, Web Retrieval,
and usability factors for mobile devices. She has been a workshop leader for
many workshops including previous WWW conferences in the Hypertext
Functionality series of workshops as well as co-chair of panels at the WWW8
conference held in Toronto.

Amanda Spink is Associate Professor at the School of Information Sciences at
the University of Pittsburgh. Her research focuses on theoretical and
applied studies of human information behavior, Web search, information
retrieval, and digital libraries studies. The National Science Foundation,
Andrew R. Mellon Foundation, NEC, IBM, Excite, AlltheWeb.com, Alta Vista and
Lockheed Martin have sponsored her research. She has published over 180
journal articles and conference papers, with many in the Journal of the
American Society for Information Science and Technology, Information
Processing and Management, Interacting with Computers, IEEE Computer, and
Internet Research, the ASIST and ISIC Conferences. She is co-author on the
forthcoming book Web Search: Public Searching of the Web.

			Workshop History

The proposed workshop follows a trail of related workshops starting with the
Hypertext Functionality workshops from HTF I (ACM ECHT'94, Edinburgh)
through HTF IX (ACM DL'00, San Antonio) and including HTF IV Hypertext on
the Web (WWW7, Brisbane) and HTFVI Hypertext on the Web (WWW8, Toronto).


------------
Richard Hill
Executive Director
American Society for Information Science and Technology
1320 Fenwick Lane, Silver Spring, MD  20910 
FAX: (301) 495-0810
Voice: (301) 495-0900
www.asis.org