[Sigdl-l] [Asis-l] Input sought: Evaluation of Music IR and Music DL systems

[Richard Hill]

[Forwarded for J. Stephen Downie.  Dick Hill]

Dear Colleagues:

Dr. Ellen Voorhees, of the National Institute of Standards and
Technology (United States), has just submitted her keynote white paper
for the Music Information Retrieval/Music Digital Library Evaluation
Workshop (Joint Conference on Digital Libraries 2002, 18 July 2002). I
am pre-releasing Dr. Voorhees' excellent discussion of the background of
the Text REtrival Conference (TREC) and its potential applicability as a
model for MIR/MDL evaluation. I hope that the early release of the paper
will prompt responses from all those interested in MIR/MDL evaluation
issues in particular, and IR/DL evaluation in general.

Whither Music IR Evaluation Infrastructure: Lessons to be Learned from
TREC
http://music-ir.org/evaluation/voorhess.pdf
also at:
http://www.itl.nist.gov/iaui/894.02/works/papers/jcdl_wkshop.pdf

If anything in Dr. Voorhees paper prompts a comment or an argument or a
suggestion, please share it with the group or with me personally at
jdownie at uiuc.edu. Even if you do not have a compelling interest in
MIR/MDL issues, but have some experience in other evaluation projects, I
both welcome and encourage your feedback.

If possible, I would like to gather up some of the salient responses for
inclusion as an appendix to handouts that will be given to participants
of this workshop and the upcoming Panel on MIR evaluation at the
International Conference on Music Information Retrieval (ISMIR 2002,
Paris), etc. to aid in our discussions. So, if you would be so kind as
to indicate that it would be acceptable to you that we reproduce your
comments, etc., I would appreciate it greatly.

Cheers, and I hope all is well with each of you.
J. Stephen Downie

Background URLS:

Conference and Workshop URLS
http://www.ohsu.edu/jcdl/
http://www.ohsu.edu/jcdl/ws.html#W4

Please visit the following URLs for other important details:

http://music-ir.org/MIR_MDL_evaluation.html
http://music-ir.org/JCDL_Workshop_Info.html
-- 
**********************************************************
    "Research funding makes the world a better place"
**********************************************************
J. Stephen Downie, PhD
Assistant Professor,
Graduate School of Library and Information Science; and,
Fellow, National Center for Supercomputing Applications (2000-01)
University of Illinois at Urbana-Champaign
(217) 351-5037
_______________________________________________
Asis-l mailing list
Asis-l at asis.org
http://mail.asis.org/mailman/listinfo/asis-l

Executive Director
American Society for Information Science and Technology
1320 Fenwick Lane, Suite 510
Silver Spring, MD  20910
FAX: (301) 495-0810
PHONE: (301) 495-0900

http://www.asis.org