Lange T. "The imprecise science of evaluating scholarly performance - Utilizing broad quality categories for an assessment of business and management journals" EVALUATION REVIEW 30 (4): 505-532 AUG 2006

[Eugene Garfield]

Thomas Lange : thomas.lange at aut.ac.nz

Title: The imprecise science of evaluating scholarly performance - 
Utilizing broad quality categories for an assessment of business and 
management journals 

Author(s): Lange T (Lange, Thomas) 

Source: EVALUATION REVIEW 30 (4): 505-532 AUG 2006 

Document Type: Article 
Language: English 
Cited References: 60      Times Cited: 0      
  
Abstract: In a growing number of countries, government-appointed assessment 
panels develop ranks on the basis of the quality of scholarly outputs to 
apportion budgets in recognition of evaluated performance and to justify 
public funds for future R&D activities. When business and management 
journals are being grouped in broad quality categories, a recent study has 
noted that this procedure teas placing the same journals in essentially the 
same categories. Drawing on journal quality categorizations by several 
German- and English-speaking business departments and academic 
associations, the author performs nonparametric tests and correlations to 
analyze whether this claim can be substantiated. In particular, he examines 
the ability of broad quality categorizations to add value to governmental, 
administrative, and academic decision making by withstanding the criticism 
traditionally levied at research quality assessments. 


Addresses: Lange T (reprint author), Auckland Univ Technol, Fac Business, 
Auckland, New Zealand
Auckland Univ Technol, Fac Business, Auckland, New Zealand

Publisher: SAGE PUBLICATIONS INC, 2455 TELLER RD, THOUSAND OAKS, CA 91320 
USA 

Subject Category: SOCIAL SCIENCES, INTERDISCIPLINARY 
IDS Number: 065IB 

ISSN: 0193-841X 

CITED REFERENCES:
*ALL CONS GROUP
INT COMP RES QUAL NZ : 2004   

 *COMM AUSTR
RES QUAL FRAM ASS QU : 2005   

 *HIGH ED AUTH
PROGR RES 3 LEV INS : 2003   

 *ORG EC COOP DEV
FRASC MAN PROP STAND : 2002   

 *TERT ED COMM
PERF BAS RES FUND EV : 2004   

 ALEXANDER JC
RELATIVE SIGNIFICANCE OF JOURNALS, AUTHORS, AND ARTICLES CITED IN FINANCIAL 
RESEARCH
JOURNAL OF FINANCE 49 : 697 1994   

 BALLAS A
Exploring diversity in accounting through faculty journal perceptions
CONTEMPORARY ACCOUNTING RESEARCH 20 : 619 2003 
  
 BAVELAS JB
SOCIAL-PSYCHOLOGY OF CITATIONS
CANADIAN PSYCHOLOGICAL REVIEW-PSYCHOLOGIE CANADIENNE 19 : 158 1978 
  
 BEED C
Measuring the quality of academic journals: The case of economics
JOURNAL OF POST KEYNESIAN ECONOMICS 18 : 369 1996   

 BLACKBURN RS
ORGANIZATIONAL-BEHAVIOR - WHOM DO WE TALK TO AND WHO TALKS TO US
JOURNAL OF MANAGEMENT 16 : 279 1990   

 BOURKE P
EVALUATING U RES BRI : 1997   

 BROWN LD
CONTEMP ACCOUNT RES 11 : 223 1994   

 BROWN LD
REV QUANTITATIVE FIN 20 : 291 2003   

 CHAN KC
FINANC MANAGE : 131 2002   

 COE R
AACSB B 1 : 23 1969   

 COE R
EVALUATING THE MANAGEMENT JOURNALS - A 2ND LOOK
ACADEMY OF MANAGEMENT JOURNAL 27 : 660 1984   

 COLLIN SO
BRIT J MANAGE 7 : 141 1996   

 CONOVER WJ
PRACTICAL NONPARAMET : 2003   

 CROOM DL
DANGERS IN USE OF SCIENCE CITATION INDEX
NATURE 227 : 1173 1970   

 CUDD M
FINANCIAL REV 23 : 117 1988   

 DUNDEN G
ATLANTIC ECON J 21 : 1 1994   

 DUNN OJ
MULTIPLE COMPARISONS USING RANK SUMS
TECHNOMETRICS 6 : 241 1964   

 EXTEJT MM
THE BEHAVIORAL-SCIENCES AND MANAGEMENT - AN EVALUATION OF RELEVANT JOURNALS
JOURNAL OF MANAGEMENT 16 : 539 1990   

 GARFIELD E
SOCIAL SCI CITATION 17 : 1991   

 GOMEZMEJIA LR
DETERMINANTS OF FACULTY PAY - AN AGENCY THEORY PERSPECTIVE
ACADEMY OF MANAGEMENT JOURNAL 35 : 921 1992   

 HARZING A
J QUALITY LIST : 2000   

 HASSELBACK J
MCGRAW HILL DIRECTOR : 1996   

 HASSELBACK JR
ISSUES ACCOUNTING ED 10 : 269 1995   

 HENDERSON G
ACROSS-DISCIPLINE JOURNAL AWARENESS AND EVALUATION - IMPLICATIONS FOR THE 
PROMOTION AND TENURE PROCESS
JOURNAL OF ECONOMICS AND BUSINESS 42 : 325 1990  
 
 HOLMSTROM B
MORAL HAZARD AND OBSERVABILITY
BELL JOURNAL OF ECONOMICS 10 : 74 1979   

HOLSAPPLE CW
A CITATION ANALYSIS OF BUSINESS COMPUTING RESEARCH JOURNALS
INFORMATION & MANAGEMENT 25 : 231 1993   

 HULT G
J MARKETING ED 19 : 37 1997  
 
 HUSTAD TP
Untitled
JOURNAL OF PRODUCT INNOVATION MANAGEMENT 14 : 157 1997 
  
 JOBBER D
INT J RES MARK 5 : 137 1988   

 JOHNSON JL
JOURNAL INFLUENCE IN THE FIELD OF MANAGEMENT - AN ANALYSIS USING SALANCIK 
INDEX IN A DEPENDENCY NETWORK
ACADEMY OF MANAGEMENT JOURNAL 37 : 1392 1994
   
 KENDALL MG
The problem of m rankings
ANNALS OF MATHEMATICAL STATISTICS 10 : 275 1939 
  
 KRUSKAL WH
USE OF RANKS IN ONE-CRITERION VARIANCE ANALYSIS
JOURNAL OF THE AMERICAN STATISTICAL ASSOCIATION 47 : 583 1952   

 LANE DM
HYPERSTAT ONLINE INT : 2003 
  
 LOFT A
EUROPEAN ACCOUNTING 11 : 43 2002   

 LUKKA K
Is accounting a global or a local discipline? Evidence from major research 
journals
ACCOUNTING ORGANIZATIONS AND SOCIETY 21 : 755 1996 
  
 LYALL C
Assessing end-use relevance of public sector research organisations
RESEARCH POLICY 33 : 73 2004   

 MACROBERTS MH
PROBLEMS OF CITATION ANALYSIS - A CRITICAL-REVIEW
JOURNAL OF THE AMERICAN SOCIETY FOR INFORMATION SCIENCE 40 : 342 1989   

 MAY KO
SCIENCE MAY : 890 1967 
  
 PANAZZO F
ACCOUNT ORG SOC 22 : 447 1997
   
 PARNELL JA
MIDATLANTIC J BUSINE 33 : 69 1997 
  
 POLONSKY MJ
J MARKETING ED 21 : 181 1999  
 
 ROBERTS G
REV RES ASSESSMENT : 2003   

 SCHIERMEIER Q
NATURE 18 : 260 2004  
 
 SCHMIDT RC
Managing Delphi surveys using nonparametric statistical techniques
DECISION SCIENCES 28 : 763 1997   

 SCHOFIELD PJ
SIIMPLE STAT APPROAC : 2005   

 SEGLEN PO
CAUSAL RELATIONSHIP BETWEEN ARTICLE CITEDNESS AND JOURNAL IMPACT
JOURNAL OF THE AMERICAN SOCIETY FOR INFORMATION SCIENCE 45 : 1 1994 
  
 SMITH J
J ACCOUNTING ED 16 : 497 1998   

 SOTERIOU AC
J OPER MANAG 17 : 225 1999 
  
 THEOHARAKIS V
Perceptual differences of marketing journals: A worldwide perspective
MARKETING LETTERS 13 : 389 2002   

 TUKEY JW
COLLECTED WORKS JW T : 1994   

 VANFLEET DD
A theoretical and empirical analysis of journal rankings: The case of 
formal lists
JOURNAL OF MANAGEMENT 26 : 839 2000   

 VASTAG G
Journal characteristics, rankings and social acculturation in operations 
management
OMEGA-INTERNATIONAL JOURNAL OF MANAGEMENT SCIENCE 30 : 109 2002 
  
 WALLIS WA
The correlation ratio for ranked data
JOURNAL OF THE AMERICAN STATISTICAL ASSOCIATION 34 : 533 1939
   
 WOODWARD AM
J DOCUMENTATION DEC : 290 1976   

 ZAR JH
BIOSTATISTICAL ANAL : 1996