Times Higher Education Rankings and the Rise of Asia

The latest Times Higher Education (THE) Rankings were released a few weeks ago.  You probably saw it linked to headlines that read, "The Rise of Asia", or some such thing.  That’s probably not news; the THE has run with “rise of Asia” stories every year since 2011 when they re-did their methodology following their divorce from Quacquarelli Symonds (QS).

But was there really a rise of Asia this year?   Look closely at the changes in the ranking positions from 2013-14.

Table 1: Asian Universities and the THE top 200, 2013-14

As Table 1 shows, whatever evidence exists for a “rise of Asia” narrative clearly does not lie in the top 50.  Tokyo and Hong Kong were unchanged in their position.  Peking University rose one place and National University of Singapore rose three, but Tsinghua fell one place and Seoul National fell six.  At best this is a “no change” for the continent.

Below that, we see a mix of good and bad, at least among East Asian universities.  Nearly all the Japanese universities saw double-digit falls in places, as did National Taiwan University and Chinese University of Hong Kong.  In Korea, Postech, a bright star in many recent years, fell six places from 60^th to 66^th, while Yonsei fell out of the top 200 altogether.  The only East Asian institutions which in the previous year ranked between 50 and 200 saw a rise in their ranked position were KAIST and HKUST.  Offsetting this performance somewhat was the rise into the top 200 of City University of Hong Kong (192^nd), Fudan University (193^rd) and Sungkyunkwan University (148^th).  So while there was a net gain of 2 institutions in the top 200, the average position of East Asian universities fell somewhat.  That’s at best a mixed picture, not a gain.

So where does the “rise of Asia” story come from?  Oddly enough it’s from Turkey (which, like Israel, is often considered as being European rather than Asian - at least if membership in UEFA and Eurovision is anything to go by). And how did Turkey manage to do so well this year?    Well, for that you need to check in with my friend Richard Holmes, who runs the University Ranking Watch blog.  He points out that a single paper ("Observation of a new boson..." in Physics Letters B, which announced the confirmation of the Higgs Boson, and which immediately attracted several thousand citations) was responsible for most of the movement in this year's rankings. 

Because the paper had over 2,800 co-authors (including from those suddenly big Turkish universities), and because THE doesn't fractionally count multiple-authored articles, and because THE's methodology gives tons of bonus points to universities located in countries where scientific publications are low, this absolutely blew some schools' numbers into the stratosphere.  Other examples of this are Scuola Normale di Pisa, which came out of nowhere to be ranked 65th in the world, or Federica Santa Maria Technical University in Chile, which somehow became the 4th ranked university in Latin America.

An honest analysis of the results would therefore have shown this year's "rise of Asia" story was almost entirely due to the fact that a few of the 2,800 co-authors on the Higgs-Boson paper happened to work in Turkey.  The fact that it chose to continue with a simplistic “Asia rampant” storyline was therefore disappointing.

Commendably, the THE has recently started public consultations on how to improve its rankings.  Whatever its final result, it should definitely fix the sensitivity of its model to occasional freak results such as the ones generated this year by the “Observation of a new boson…” paper.  Failure to do so will lead to more freak results such as this one.