Is multidisciplinary research more highly cited? A macro-level study

Mon Oct 13 06:20:46 EDT 2008

Dear Jonathan,

I mean something simple using your data. You could for example plot the
citation scores of your "interdisciplinary" papers in decreasing order of
being-citedness (one expects a Lotka-type distribution), and then add as a
second curve the impact factors of the journals. One could then get an
impression whether the noted effect could maily be caused by the upper end
of the curve (interdisciplinary articles cited more than average in
disciplinary journals) or the long tail of "interdisciplinary papers" in the
"carbage can".

But I see immediately that one may have to differentiate this for fields of
science and then it easily becomes more complex. Nevertheless, I would be
interested to know in which journals the top-100 (in terms of citation
rates) of your "interdisciplinary" papers were published. Is that in the
report?

Best, Loet

On Mon, Oct 13, 2008 at 11:39 AM, Jonathan Adams <
jonathan.adams at evidence.co.uk> wrote:

> Adminstrative info for SIGMETRICS (for example unsubscribe):
> http://web.utk.edu/~gwhitney/sigmetrics.html
>
> I think Michel Zitt drew attention to this when he pointed out that the
> 'zoom' with which you look at an item makes a difference to its citation
> calibration.  Context is all!
> And I agree: there are further tests to elaborate the findings and the
> initial hypotheses that emerge.  So, we need a sponsor!
> Jonathan Adams
>
> Evidence Ltd
> 103 Clarendon Road, Leeds LS2 9DF, UK
> t/ +44 (0) 113 384 5680
>
>
> -----Original Message-----
> From: ASIS&T Special Interest Group on Metrics
>  [mailto:SIGMETRICS at listserv.utk.edu] On Behalf Of Loet Leydesdorff
> Sent: 13 October 2008 10:05
>
> Dear Jonathan,
>
> If both of you are right -- which I have no reason to doubt -- than the
> aggregation at the journal level would make the difference. In that
> case,
> this would be a nice example of the so-called ecological fallacy: what
> one
> can see at the level of the wood, one cannot see at the level of
> individual
> trees, and vice versa (Robertson, 1950).
>
> Given the urgency of the matter in the UK, it may be worth testing this
> hypothesis: if you would reorganize your sample in terms of the journals
> involved, would then ...? Or, in other words, would interdisciplinary
> articles in disciplinary journals do better than average?
>
> Best wishes,
>
>
> Loet
>
> ________________________________
>
> Loet Leydesdorff
> Amsterdam School of Communications Research (ASCoR),
> Kloveniersburgwal 48, 1012 CX Amsterdam.
> Tel.: +31-20- 525 6598; fax: +31-20- 525 3681
> loet at leydesdorff.net ; http://www.leydesdorff.net/
>
>
>
> > -----Original Message-----
> > From: ASIS&T Special Interest Group on Metrics
> > [mailto:SIGMETRICS at listserv.utk.edu] On Behalf Of Jonathan Adams
> > Sent: Monday, October 13, 2008 10:40 AM
> > To: SIGMETRICS at listserv.utk.edu
> > Subject: [SIGMETRICS] Is multidisciplinary research more
> > highly cited? A macro-level study
> >
> > Adminstrative info for SIGMETRICS (for example unsubscribe):
> > http://web.utk.edu/~gwhitney/sigmetrics.html
> >
> > People may also be interested in Evidence Ltd's earlier study on
> > interdisciplinary research for the Higher Education Funding
> > Council for
> > England, as part of the background development of the Research
> > Excellence Framework.
> > http://www.hefce.ac.uk/pubs/rdreports/2007/rd19_07/
> > This report is a background document to HEFCE's consultation on
> > proposals for the Research Excellence Framework ('Research Excellence
> > Framework: consultation on the assessment and funding of higher
> > education research post-2008', HEFCE 2007/34).
> > We set out to develop an acceptable indicator of 'interdisciplinarity'
> > at the article level, rather than looking at articles in
> > multidisciplinary journals.  The report therefore considers how
> > interdisciplinary research could be defined for the purposes of
> > bibliometric analysis, and whether such research is
> > systematically cited
> > less often and could potentially be disadvantaged in a
> > bibliometrics-based system of research assessment.
> > The Evidence study found that there is no strong case for research
> > outputs to be treated differently for the purposes of research
> > assessment on the grounds of interdisciplinarity, but advises that
> > bibliometric analysis of such outputs should be carried out
> > carefully to
> > ensure they are treated appropriately.
> > Of course, the way in which we categorise more or less
> > interdisciplinary
> > research is important.  We exposed the methodology to senior
> > researchers
> > who had worked on previous RAE panels.  They agreed that our working
> > definition seemed to make intuitive sense in the context of their
> > (various) fields - which include social science.
> > Regards
> > Jonathan Adams
> >
> > Evidence Ltd
> > 103 Clarendon Road, Leeds LS2 9DF, UK
> > t/ +44 (0) 113 384 5680
> >
> > Registered in England No. 4036650
> > VAT Registration: GB 758 4671 85
> >
> > Please note that Evidence Ltd does not enter into any form of contract
> > via this medium, nor is our staff authorised to do so on our behalf.
> > -----Original Message-----
> > From: ASIS&T Special Interest Group on Metrics
> > [mailto:SIGMETRICS at LISTSERV.UTK.EDU] On Behalf Of Jonathan Levitt and
> > Mike Thelwall
> > Sent: 13 October 2008 00:23
> > To: SIGMETRICS at LISTSERV.UTK.EDU
> > Subject: [SIGMETRICS] Is multidisciplinary research more
> > highly cited? A
> > macro-level study
> >
> > Adminstrative info for SIGMETRICS (for example unsubscribe):
> > http://web.utk.edu/~gwhitney/sigmetrics.html
> >
> > Levitt, J.M. and Thelwall, M. (2008). Is multidisciplinary
> > research more
> >
> > highly cited? A macro-level study. Journal of the American
> > Society for
> > Information Science and Technology, 59(12), 1973-1984.
> >
> > Inter-disciplinary collaboration is a major goal in research policy.
> > This
> > study uses citation analysis to examine diverse subjects in
> > the Web of
> > Science and Scopus to ascertain whether, in general, research
> > published
> > in
> > journals classified in more than one subject is more highly
> > cited than
> > research published in journals classified in a single
> > subject. For each
> > subject the study divides the journals into two disjoint sets called
> > Multi
> > and Mono: Multi consists of all journals in the subject and
> > at least one
> >
> > other subject, whereas Mono consists of all journals in the
> > subject and
> > in
> > no other subject. The main findings are: (a) For social
> > science subject
> > categories in both the Web of Science and Scopus, the average
> > citation
> > levels of articles in Mono and Multi are very similar, and (b) For
> > Scopus
> > subject categories within Life Sciences, Health Sciences, and
> > Physical
> > Sciences, the average citation level of Mono articles is roughly twice
> > that
> > of Multi articles. Hence one cannot assume that, in general, multi-
> > disciplinary research will be more highly cited, and the converse is
> > probably true for many areas of science. A policy implication is that,
> > at
> > least in the sciences, multi-disciplinary researchers should not be
> > evaluated by citations on the same basis as mono-disciplinary
> > researchers.
> >
> > Reported in the Times Higher Education (REF could penalise
> > those working
> >
> > across disciplines,
> > http://www.timeshighereducation.co.uk/story.asp?
> > sectioncode=26&storycode=403796&c=2)
> >
> > Jonathan Levitt and Mike Thelwall
> >
>

-- 
Loet Leydesdorff
Amsterdam School of Communications Research (ASCoR)
Kloveniersburgwal 48, 1012 CX Amsterdam
Tel.: +31-20- 525 6598; fax: +31-20- 525 3681
loet at leydesdorff.net ; http://www.leydesdorff.net/
---------------------------------------
Now available: The Knowledge-Based Economy: Modeled, Measured, Simulated,
385 pp.; US$ 18.95;
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.asis.org/pipermail/sigmetrics/attachments/20081013/f905475f/attachment.html>