Is multidisciplinary research more highly cited? A macro-level study

[Loet Leydesdorff]

Dear Jonathan, 

If both of you are right -- which I have no reason to doubt -- than the
aggregation at the journal level would make the difference. In that case,
this would be a nice example of the so-called ecological fallacy: what one
can see at the level of the wood, one cannot see at the level of individual
trees, and vice versa (Robertson, 1950).

Given the urgency of the matter in the UK, it may be worth testing this
hypothesis: if you would reorganize your sample in terms of the journals
involved, would then ...? Or, in other words, would interdisciplinary
articles in disciplinary journals do better than average? 

Best wishes,


Loet

________________________________

Loet Leydesdorff 
Amsterdam School of Communications Research (ASCoR), 
Kloveniersburgwal 48, 1012 CX Amsterdam. 
Tel.: +31-20- 525 6598; fax: +31-20- 525 3681 
loet at leydesdorff.net ; http://www.leydesdorff.net/ 

 

> -----Original Message-----
> From: ASIS&T Special Interest Group on Metrics 
> [mailto:SIGMETRICS at listserv.utk.edu] On Behalf Of Jonathan Adams
> Sent: Monday, October 13, 2008 10:40 AM
> To: SIGMETRICS at listserv.utk.edu
> Subject: [SIGMETRICS] Is multidisciplinary research more 
> highly cited? A macro-level study
> 
> Adminstrative info for SIGMETRICS (for example unsubscribe):
> http://web.utk.edu/~gwhitney/sigmetrics.html
> 
> People may also be interested in Evidence Ltd's earlier study on
> interdisciplinary research for the Higher Education Funding 
> Council for
> England, as part of the background development of the Research
> Excellence Framework.
> http://www.hefce.ac.uk/pubs/rdreports/2007/rd19_07/
> This report is a background document to HEFCE's consultation on
> proposals for the Research Excellence Framework ('Research Excellence
> Framework: consultation on the assessment and funding of higher
> education research post-2008', HEFCE 2007/34).
> We set out to develop an acceptable indicator of 'interdisciplinarity'
> at the article level, rather than looking at articles in
> multidisciplinary journals.  The report therefore considers how
> interdisciplinary research could be defined for the purposes of
> bibliometric analysis, and whether such research is 
> systematically cited
> less often and could potentially be disadvantaged in a
> bibliometrics-based system of research assessment.
> The Evidence study found that there is no strong case for research
> outputs to be treated differently for the purposes of research
> assessment on the grounds of interdisciplinarity, but advises that
> bibliometric analysis of such outputs should be carried out 
> carefully to
> ensure they are treated appropriately.
> Of course, the way in which we categorise more or less 
> interdisciplinary
> research is important.  We exposed the methodology to senior 
> researchers
> who had worked on previous RAE panels.  They agreed that our working
> definition seemed to make intuitive sense in the context of their
> (various) fields - which include social science.
> Regards
> Jonathan Adams
> 
> Evidence Ltd
> 103 Clarendon Road, Leeds LS2 9DF, UK
> t/ +44 (0) 113 384 5680
> 
> Registered in England No. 4036650
> VAT Registration: GB 758 4671 85
> 
> Please note that Evidence Ltd does not enter into any form of contract
> via this medium, nor is our staff authorised to do so on our behalf.
> -----Original Message-----
> From: ASIS&T Special Interest Group on Metrics
> [mailto:SIGMETRICS at LISTSERV.UTK.EDU] On Behalf Of Jonathan Levitt and
> Mike Thelwall
> Sent: 13 October 2008 00:23
> To: SIGMETRICS at LISTSERV.UTK.EDU
> Subject: [SIGMETRICS] Is multidisciplinary research more 
> highly cited? A
> macro-level study
> 
> Adminstrative info for SIGMETRICS (for example unsubscribe):
> http://web.utk.edu/~gwhitney/sigmetrics.html
> 
> Levitt, J.M. and Thelwall, M. (2008). Is multidisciplinary 
> research more
> 
> highly cited? A macro-level study. Journal of the American 
> Society for 
> Information Science and Technology, 59(12), 1973-1984.
> 
> Inter-disciplinary collaboration is a major goal in research policy.
> This 
> study uses citation analysis to examine diverse subjects in 
> the Web of 
> Science and Scopus to ascertain whether, in general, research 
> published
> in 
> journals classified in more than one subject is more highly 
> cited than 
> research published in journals classified in a single 
> subject. For each 
> subject the study divides the journals into two disjoint sets called
> Multi 
> and Mono: Multi consists of all journals in the subject and 
> at least one
> 
> other subject, whereas Mono consists of all journals in the 
> subject and
> in 
> no other subject. The main findings are: (a) For social 
> science subject 
> categories in both the Web of Science and Scopus, the average 
> citation 
> levels of articles in Mono and Multi are very similar, and (b) For
> Scopus 
> subject categories within Life Sciences, Health Sciences, and 
> Physical 
> Sciences, the average citation level of Mono articles is roughly twice
> that 
> of Multi articles. Hence one cannot assume that, in general, multi-
> disciplinary research will be more highly cited, and the converse is 
> probably true for many areas of science. A policy implication is that,
> at 
> least in the sciences, multi-disciplinary researchers should not be 
> evaluated by citations on the same basis as mono-disciplinary
> researchers.
> 
> Reported in the Times Higher Education (REF could penalise 
> those working
> 
> across disciplines, 
> http://www.timeshighereducation.co.uk/story.asp?
> sectioncode=26&storycode=403796&c=2)
> 
> Jonathan Levitt and Mike Thelwall
>