Manipulating Citation Rankings?
The possession by Saudi Arabia’s King Abdulaziz University of more highly cited researchers than almost any other university in the world raises questions about institutions’ ability to manipulate global rankings.
This is the view of two researchers who last week posted on the arXiv preprint server a paper that ranks universities by the number of highly cited researchers who list them as affiliations.
The analysis was done by Lutz Bornmann, a bibliometrician working at the headquarters of the Max Planck Society in Munich, and Johann Bauer of the Max Planck Institute of Biochemistry in Martinsried, Germany. They examined the addresses on papers written by the world’s most highly cited researchers between 2002 and 2012, as revealed by Thomson Reuters last month.
According to their paper, “Which of the World’s Institutions Employ the Most Highly Cited Researchers? An Evaluation of the Data From Highlycited.com,” the University of California system is listed as the primary affiliation of 179 highly cited researchers, followed by Harvard University with 106 and Stanford University with 56.
However, some authors also list secondary academic affiliations. When ranked according to how many highly cited researchers list them as either a primary or a secondary affiliation, the University of California remains top, with 198, but King Abdulaziz University leaps from nowhere to second, with 163.
Noting that the Academic Ranking of World Universities, often known as the Shanghai Jiao Tong rankings, takes into account an institution’s proportion of highly cited researchers, the paper says that “the results for King Abdulaziz illustrate that university rankings can be manipulated.” To counteract this, it suggests, rankings should take only primary affiliations into account. Times Higher Education’s World University Rankings also use citations to indicate a university’s research influence, among a total of 13 indicators, but they examine each institution’s citations per paper (normalized for subject), not the proportion of highly cited researchers.
According to another unpublished analysis of the Thomson Reuters data by Yves Gingras, a professor at the University of Montreal’s Centre Interuniversitaire de Recherche sur la Science et la Technologie, Saudi Arabia is the country with by far the highest proportion (82 percent) of highly cited researchers who list its institutions only as secondary overseas affiliations. The second highest is South Africa, with 45 percent. King Abdulaziz is listed as a secondary affiliation by 122 researchers, compared with 27 for the next most commonly listed institution, Harvard.
Professor Gingras told Times Higher Education that a 2011 investigation by the journal Science revealed that some Saudi Arabian universities, including King Abdulaziz, offer highly cited researchers lucrative adjunct professorships, with minimal requirements for them to be physically present, in return for being listed by them as a secondary affiliation.
Over all, of the 3,215 researchers in Thomson Reuters’ list, 702 have more than one affiliation, with 42 percent listing an overseas institution. In Britain, 36 percent of highly cited researchers have a secondary affiliation.
“The minimum that should be done by universities is to make sure the professors they pay full time put only their own addresses on their papers. This would control the traffic of addresses by unscrupulous institutions,” Gingras said.
King Abdulaziz did not respond to a request for comment.
Search for Jobs