Isidro F. Aguillo, Bego ña Granadino, Jose Luis Ortega and Jose Antonio Prieto "What the Internet says about Science: Universities can be ranked based on web indicators" The Scientist | Volume 19 | Issue 14 | Page 10 | Jul. 18, 2005

[Eugene Garfield]

E-mail: Isidro F. Aguillo [isidro at cindoc.csic.es],

FROM:
The Scientist
Volume 19 | Issue 14 | Page 10 | Jul. 18, 2005

UPFRONT | OPINION

TITLE    :  What the Internet says about Science:
            Universities can be ranked based on web indicators

AUTHORS:    Isidro F. Aguillo, Begoña Granadino, Jose Luis Ortega and
            Jose Antonio Prieto


The Web has changed the way in which many researchers access scientific
information, conduct research, communicate their findings, and share data.
There is now a need to assess the impact of Web publication in order to
promote wider and better use of this new medium.1 Recent attempts have been
made to go beyond the strict use of bibliometric indicators. Shanghai Jiao
Tong University has published a ranking2 of the top 500 universities, in
which numbers of publications and citations were combined with other
criteria such as institution size or the number of Nobel prizes awarded to
alumni.

The Web offers advantages as institutions represent "natural units," with
their own institutional domains that mark their presence on the Internet.
Since most institutions have a specific Internet domain or subdomain for
all their Web pages,3 quantitative data can be extracted using specifically
designed crawlers, or the robots of the major search engines.

The contents of these institutional Web sites might include not only final
papers or preprints, but also valuable information on other aspects of
their scientific activities. Raw data, teaching materials, slides produced
for meetings or conferences, in-house software, graphs, media files, and
even administrative information might be useful to pupils, colleagues, and
partners worldwide. Since the Web is ubiquitous, a wider audience is
possible when publication is electronic; this could include readers in
developing countries whose access to scientific publications can be very
restricted. The Web is cheaper than the printed word and can provide
information that paper sources could never contain; for example, large
amounts of data, complex and dynamic graphics, or even interactive systems.
Finally, the Web is a hypertext-interlinked system, and although the
motivations for linking far exceed traditional "citation for recognition,"
careful use of citation analysis techniques may still be possible.

Web indicators are now becoming important in the quantitative analysis of
science,4 but a global system involving the major universities and research
institutions has yet to be developed. To fill this gap, we designed a
combined assessment model for ranking the institutional domains of
universities worldwide based on "Web presence" indicators. Three different
features of these domains were assessed: the size of their Web presence
(measured by the number of Web pages), visibility (reflected by the number
of in-links from pages external to the domain), and the number of "rich"
files available (the number of downloadable files in advanced formats such
as Adobe Acrobat – pdf, PostScript – ps, MS Word – doc, MS PowerPoint –
ppt, and MS Excel – xls).

Table 1
Rank   University                          Size  Visibility   Rich Files
1   Stanford University                      3      3            1
2   University of California,Berkeley        4      2            3
3   Massachusetts Institute of Technology    5      1            6
4   Harvard University                       1      4            4
5   Pennsylvania State University            2      9            2
6   Univ. of Illinois,Urbana-Champaign       7      7            7
7   University of Michigan                   6      6           16
8   Cornell University                      13      5            9
9   University of Texas,Austin               9      8           10
10  University of Wisconsin, Madison        10     10           12

While evaluations were made in absolute terms, the fact that research and
other activities involve specific file types was taken into account. The
number of in-links to these organizations' Web sites was used to establish
the visibility of their content to third parties, including other academic
and scientific organizations, government authorities, and companies.

Data were collected using the major search engines. The full results are
available from "Ranking of World Universities in the Web"
[http://www.webometrics.info], where the first 1,000 universities are
listed according to Web criteria. Preliminary analysis shows that most
productive research-oriented universities are among the leaders in the list
(Table 1).

Table 2

Country           Top 100    Top 200    Top 500
USA               66          104        208
Canada             7           16         26
Germany            5           21         52
United Kingdom     5           12         37
Sweden             3            7         13
Norway             3            3          4
Australia          2            7         19
Switzerland        2            5          7
Finland            2            2          8
Austria            2            2          8
Netherlands        1            6          8
Brazil             1            2          6
Mexico             1            1          3
Italy                           2         13
France                          2         10


A significant positive correlation was found between the Web list ranks and
the Shanghai Jiao Tong University list. Moreover, a large number of
technologically oriented institutions were well-positioned by the
Webometric indicators. The universities of developing countries (especially
of the larger nations) appeared in competitive positions with regard to
those of the developed world (Table 2).

There are still several technical and methodological problems to overcome,
mostly related with the search engine bias. More relevant, there are
several shortcomings and caveats related to the use of Web indicators for
assessing visibility and impact. These are not yet developed enough to
compare directly to bibliometric ones, but we intend to use non-Webometric
indicators in our rankings to allow a direct comparison of the different
approaches.

Isidro F. Aguillo [isidro at cindoc.csic.es], Begoña Granadino, Jose Luis
Ortega, and Jose Antonio Prieto are researchers in the Internet Lab at the
Center for Scientific Information and Documentation (CINDC), Spanish Higher
Council for Scientific Research, in Madrid, Spain.

References

1. M Thelwall et al, http://cybermetrics.wlv.ac.uk/AoIRASIST/index.html
Proceedings of the AoIR-ASIST 2004 Workshop on Web Science Research Methods
2004.
2. NC Liu et al, "Academic Ranking of World Universities – 2004,"
http://ed.sjtu.edu.cn/ranking.htm Institute of Higher Education, Shanghai
Jiao Tong University 2004.
3. IF Aguillo STM information on the Web and the development of new
Internet R&D databases and indicators Proceedings Online Information
Meeting 98, London, Learned information 1998, 239-43.
4. M Thelwall et al, "Webometrics," Annual Review of Information Science
and Technology (Edited by: Cronin B). Medford, NY: Information Today, Inc
2005, 39: 81-135.