[Sigmet-officers] Paper contest: appointment of reviewers

[Judit Bar-Ilan]

Dear All,

I am willing to be a fixed reviewer, but six submissions per reviewer is a
bit too much. Four papers per reviewer is much more reasonable. So please
rethink the process.
Also, what is the time frame for reviewing?

Regards
Judit

On Sat, Mar 12, 2011 at 12:00 PM, <sigmet-officers-request at asis.org> wrote:

> Send Sigmet-officers mailing list submissions to
>        sigmet-officers at mail.asis.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
>        http://mail.asis.org/mailman/listinfo/sigmet-officers
> or, via email, send a message with subject or body 'help' to
>        sigmet-officers-request at mail.asis.org
>
> You can reach the person managing the list at
>        sigmet-officers-owner at mail.asis.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of Sigmet-officers digest..."
>
>
> Today's Topics:
>
>   1. Re: Paper contest ? appointment of reviewers (Jonathan Levitt)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Fri, 11 Mar 2011 15:55:50 -0800 (PST)
> From: Jonathan Levitt <jonathan at levitt.net>
> Subject: Re: [Sigmet-officers] Paper contest ? appointment of
>        reviewers
> To: sigmet-officers at asis.org
> Message-ID: <86400.85547.qm at web1205.biz.mail.gq1.yahoo.com>
> Content-Type: text/plain; charset="utf-8"
>
> Hi all,
> ?
> Following on from Diermar?s posting, here are my initial suggestions
> regarding reviewers:
> (a)?????? Any SIG/MET officer with a Ph.D. qualifies to be a reviewer,
> irrespective of whether they have had previous experience of reviewing.
> (b)????? Establish how many qualifying SIG/MET officers are willing to
> review, before asking for external reviewers.
> (c)?????? Work out how many other people to ask to review. ?This is not
> easy as we don?t know in advance how many papers will be submitted.? However
> we can adjust the reviewing process to the number of papers submitted.?
> (d)????? One way of adjusting the reviewing process to the number of
> submissions is to have three reviewers per paper if we receive a small
> number of submissions and two reviewers per paper if we receive a small
> number of submissions. Another way of adjusting the reviewing process is for
> some reviewers to only be deployed if needed.
> (e)?????? I would be happy to be an ?optional? reviewer, i.e., to review if
> we receive a large number of papers, but not if we receive a small number of
> papers.? I suggest we quantify how many SIG/MET officers volunteering to
> review are willing to be ?optional? reviewers, before asking for external
> reviewers.
> (f)??????? Here is an example of how the reviewing process could be
> adjusted.? Suppose we have 8 reviewers, 5 ?fixed? reviewers and 3 ?optional?
> reviewers (willing to review if needed).? If we receive 10 papers, then we
> could send each of the 5 fixed reviewers 6 papers to review (30 reviews, so
> 3 reviewers per paper).? If we receive 24 papers, we could send each of the
> 8 reviewers 6 papers to review (48 reviews, so 2 reviewers per paper).?
> (g)????? The simplest way of selecting the winner and runner-up is to add
> the marks of the reviewers. ?However this is very flawed, as reviewers could
> vary enormously in the way the generosity of their marking.? ?One way of
> reducing the vagaries of markers, is to add the normalized marks (e.g., the
> number of marks divided by the average mark of the reviewer).? A more
> complicated, but I think fairer, way is to identify the strongest five or
> six articles and subject these to a second stage review.
> ?
> Please let me know what you think.
> Jonathan.
>
> --- On Wed, 9/3/11, Dietmar Wolfram <dwolfram at uwm.edu> wrote:
>
>
> From: Dietmar Wolfram <dwolfram at uwm.edu>
> Subject: Re: [Sigmet-officers] Paper contest ? appointment of reviewers
> To: "Jonathan Levitt" <jonathan at levitt.net>
> Cc: sigmet-officers at asis.org
> Date: Wednesday, 9 March, 2011, 22:46
>
>
>
> #yiv1803289635 p {margin:0;}
>
>
> Depending on the number of papers and the number of reviews per paper, we
> may only need a handful of reviewers beyond the officers, which could be
> solicited from the ISSI community, as Jonathan suggests. Would two reviews
> per submission suffice? Three reviewers are good to avoid split decisions,
> but that could add more work. Perhaps a third reviewer could be added only
> in those cases?where there are?notable differences between reviewers.
>
> Dietmar
>
>
>
>
>
>
> From: "Jonathan Levitt" <jonathan at levitt.net>
> To: sigmet-officers at asis.org
> Sent: Wednesday, March 9, 2011 12:47:57 PM
> Subject: [Sigmet-officers] Paper contest ? appointment of reviewers
>
>
>
>
>
>
> Hi,
>
> ?
>
> I think we should start discussing the appointment of reviewers, as I think
> we should ask the proposed reviewers well before April 10. Reviewing could
> be done by suitably qualified SIG/MET
>
> Officers and/or by reviewers for Infometric conferences (such as ISSI).?
> Any thoughts?
>
> ?
>
> Jonathan.
>
> ?
> _______________________________________________
> Sigmet-officers mailing list
> Sigmet-officers at mail.asis.org
> http://mail.asis.org/mailman/listinfo/sigmet-officers
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL:
> http://mail.asis.org/pipermail/sigmet-officers/attachments/20110311/e3df66da/attachment-0001.html
>
> ------------------------------
>
> _______________________________________________
> Sigmet-officers mailing list
> Sigmet-officers at mail.asis.org
> http://mail.asis.org/mailman/listinfo/sigmet-officers
>
>
> End of Sigmet-officers Digest, Vol 5, Issue 4
> *********************************************
>



-- 
Judit Bar-Ilan
Head of Department
Department of Information Science
Bar-Ilan University, Ramat Gan, 52900, Israel
Tel: 972-3-5318351 Fax: 972-3-7384027
email: barilaj at mail.biu.ac.il
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.asis.org/pipermail/sigmet-officers/attachments/20110312/e55c1260/attachment.html