[Sigmet-officers] Update on the paper contest

Thu Mar 31 13:49:59 EDT 2011

I don't know if some criteria may be left blank. If originality assessment might be problematic , then perhaps a criterion that addresses the significance of the research problem could be used instead. You could have two well-crafted submissions-- but if one is on an interesting and novel topic and the other is on a perennial topic that has been studied to death, it would be nice to acknowledge the submission that makes the larger contribution. 

A bidding feature on a conference submission system allows reviewers to see submission abstracts and then indicate which of the submissions they would be interested in reviewing. Assignments are then based on the expressed interest.  I'd be willing to join you in assigning the submissions to reviewers if no one else has done so already. 

Dietmar 

----- Original Message -----

From: "Jonathan Levitt" <jonathan at levitt.net> 
To: "SIG MET" <sigmet-officers at asis.org> 
Sent: Thursday, March 31, 2011 10:57:59 AM 
Subject: Re: [Sigmet-officers] Update on the paper contest 

Hi, 

Thanks Dietmar for your suggestions.   I have given my feedback on Cassidy’s suggestion in my last posting.   However I can amplify.   For ISSI I found it hard to assess the originality of a submission outside my core area (on on Web link analysis).   For my part I would not like to undertake additional work that is not central to our remit.   But, given that both you and Cassidy want additional criteria, I am content to include them, provided that reviewers could choose not to score for these additional criteria.   Is this possible?   

Could you pleases clarify what is a bidding feature?   It is important for things to movie quickly, especially as one reviewer has asked to be sent the papers as early as possible.   I agree that we need twp officers to be involved in this process, and volunteer to be one of these officers.   Could someone else who will give this matter high priority please volunteer to join me in this process? 

Thanks, 

Jonathan. 

--- On Thu, 31/3/11, Dietmar Wolfram <dwolfram at uwm.edu> wrote: 

From: Dietmar Wolfram <dwolfram at uwm.edu> 
Subject: Re: [Sigmet-officers] Update on the paper contest 
To: "Cassidy Sugimoto" <cassidysugimoto at gmail.com> 
Cc: "SIG MET" <sigmet-officers at asis.org> 
Date: Thursday, 31 March, 2011, 14:19 

I also like the idea of including criteria for quality and originality. 

It looks like EasyChair does not have a bidding feature. Therefore, to keep the assignment process manageable, I suggest having two officers perform this task. Having more involved could slow things down. Those interested could let Jonathan know. 

Dietmar 

From: "Cassidy Sugimoto" <cassidysugimoto at gmail.com> 
Cc: "SIG MET" <sigmet-officers at asis.org> 
Sent: Wednesday, March 30, 2011 11:20:56 PM 
Subject: Re: [Sigmet-officers] Update on the paper contest 

I know we don't want too many evaluation criteria, but I would also like to see components for the quality of the methods and the originality of the research...(and maybe "importance" or research or some other word for assessing contribution or potential to advance knowledge...)  Just my two cents--feel free to disregard. 

On Wed, Mar 30, 2011 at 3:56 PM, Jonathan Levitt < jonathan at levitt.net > wrote: 

Dear all, 

Of the ten potential reviewers, four have accepted (Kevin Boyack, Katherine McCain, Ronald Roussueau and Mike Thelwall), three declined and three have not yet replied. 

I suggest that we finalise the reviewing criteria well before the deadline.   On the basis of previous discussions, I suggest in addition to the default criteria (Overall evaluation, -3 to 3; 

Reviewer confidence, 0 to 4) we have the following criteria: 

(1)       Potential for publication (1 to 5). 

(2)       Quality of the writing (1 to 5). 

(3)       Comments on potential for publication and quality of writing. 

As the reviewers prefer an absolute score, I suggest we go for it.   We don’t have anyone to normalise, but hopefully the results will not be too skewed by not normalising. 

I have two questions: 

(a)        How are the papers assigned to reviewers? 

(b)       Is there any way of making sure that    reviewers receive their papers ASAP. 

Judit wrote “I am not sure, but I think that there is a way to send the scores to the authors - Chaoqun can probably test this.”     Chaoqun could you please find out and also how many papers have already been submitted. 

Best regards, 

Jonathan. 

_______________________________________________ 
Sigmet-officers mailing list 
Sigmet-officers at mail.asis.org 
http://mail.asis.org/mailman/listinfo/sigmet-officers 

-- 
Cassidy R. Sugimoto, PhD 
Assistant Professor 
School of Library and Information Science 
Indiana University Bloomington 
http://ella.slis.indiana.edu/~sugimoto 

_______________________________________________ 
Sigmet-officers mailing list 
Sigmet-officers at mail.asis.org 
http://mail.asis.org/mailman/listinfo/sigmet-officers 

From: "Cassidy Sugimoto" <cassidysugimoto at gmail.com> 
Cc: "SIG MET" <sigmet-officers at asis.org> 
Sent: Wednesday, March 30, 2011 11:20:56 PM 
Subject: Re: [Sigmet-officers] Update on the paper contest 

I know we don't want too many evaluation criteria, but I would also like to see components for the quality of the methods and the originality of the research...(and maybe "importance" or research or some other word for assessing contribution or potential to advance knowledge...)  Just my two cents--feel free to disregard. 

On Wed, Mar 30, 2011 at 3:56 PM, Jonathan Levitt < jonathan at levitt.net > wrote: 

Dear all, 

Of the ten potential reviewers, four have accepted (Kevin Boyack, Katherine McCain, Ronald Roussueau and Mike Thelwall), three declined and three have not yet replied. 

I suggest that we finalise the reviewing criteria well before the deadline.   On the basis of previous discussions, I suggest in addition to the default criteria (Overall evaluation, -3 to 3; 

Reviewer confidence, 0 to 4) we have the following criteria: 

(1)       Potential for publication (1 to 5). 

(2)       Quality of the writing (1 to 5). 

(3)       Comments on potential for publication and quality of writing. 

As the reviewers prefer an absolute score, I suggest we go for it.   We don’t have anyone to normalise, but hopefully the results will not be too skewed by not normalising. 

I have two questions: 

(a)        How are the papers assigned to reviewers? 

(b)       Is there any way of making sure that    reviewers receive their papers ASAP. 

Judit wrote “I am not sure, but I think that there is a way to send the scores to the authors - Chaoqun can probably test this.”     Chaoqun could you please find out and also how many papers have already been submitted. 

Best regards, 

Jonathan. 

_______________________________________________ 
Sigmet-officers mailing list 
Sigmet-officers at mail.asis.org 
http://mail.asis.org/mailman/listinfo/sigmet-officers 

-- 
Cassidy R. Sugimoto, PhD 
Assistant Professor 
School of Library and Information Science 
Indiana University Bloomington 
http://ella.slis.indiana.edu/~sugimoto 

_______________________________________________ 
Sigmet-officers mailing list 
Sigmet-officers at mail.asis.org 
http://mail.asis.org/mailman/listinfo/sigmet-officers 

-----Inline Attachment Follows----- 

_______________________________________________ 
Sigmet-officers mailing list 
Sigmet-officers at mail.asis.org 
http://mail.asis.org/mailman/listinfo/sigmet-officers 

_______________________________________________ 
Sigmet-officers mailing list 
Sigmet-officers at mail.asis.org 
http://mail.asis.org/mailman/listinfo/sigmet-officers 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.asis.org/pipermail/sigmet-officers/attachments/20110331/1e6b88d9/attachment.html 