You have /5 articles left.
Sign up for a free account or log in.
In 2013, University College Cork, in Ireland, asked its faculty members to encourage people familiar with the university to register to vote in the survey of universities conducted by Quacquarelli Symonds (QS), one of the major producers of international higher ed rankings. Many people at the time said recruiting people to vote would make the reputation reviews scientifically invalid, since the voting pool could be influenced by having more judges loyal to or familiar with one university than others. QS responded by banning universities from recruiting people to vote in its rankings surveys and suspending a system in which people could just sign up to be reviewers.
This year, Trinity College Dublin sent out letters to academics and others -- some of which were forwarded by recipients to Inside Higher Ed -- that appear to be encouraging people to register to volunteer to vote in QS rankings. Those who forwarded the letters, who asked for anonymity, said that they viewed them as violating the QS rules and that the letters seemed especially surprising coming from Trinity College Dublin, given that it is the highest-ranked institution in Ireland.
On Sunday, QS announced that it considered Trinity College Dublin "in breach" of its rules and that it could face suspension from the rankings.
Beyond the issue of Trinity College Dublin, some say the letters also show the continued vulnerability of QS rankings to being gamed by some universities.
The Trinity College Dublin letters were sent by John J. Boland, vice president and dean of research. "University rankings are also important to Trinity, and all other universities around the world, and we were pleased that the latest rankings reaffirmed Trinity College Dublin’s position as Ireland’s top university and one of the best in Europe. We know that our research successes are augmented by our collaborations with world-class academics such as you," Boland wrote to a large group of academics.
He then noted that the period in which QS and Times Higher Education conduct surveys is about to start, and Boland asked people to take the time to fill out the surveys. Further, he provided a link to the QS webpage where one can sign up to be a reviewer. It is with that link and encouragement that critics say Trinity College Dublin violated the rules.
Simona Bizzozero, a spokeswoman for QS, said via email that the ratings group was also sent the letters that were sent to Inside Higher Ed. She said QS has determined that the letters were in violation of its guidelines.
"Any attempt to overtly influence a potential survey respondent to vote in favor of a given institution is unacceptable and where identified, such respondents will [be] excluded from analysis," she said. "Further consequences for the TCD, up to and including temporary suspension from the ranking altogether, will be considered."
Boland, who sent the letters, said in an email to Inside Higher Ed that he was "surprised by the response to our letters," and that Trinity would be contacting QS today to discuss the matter.
Vulnerability Seen in System
There are three major players in international rankings of universities: QS, Times Higher Education and the Academic Ranking of World Universities (frequently called the Shanghai ranking because of its home). The latter does not use surveys but bases its rankings on such factors as number of alumni and faculty members who have won Nobel Prizes, citation indexes, and articles published in Nature and Science.
Both QS and Times Higher use surveys for portions of their various rankings, but only QS lets people nominate themselves to be surveyed.
On the QS website, the rankings organization says that in 2013, as a result of "noticeable attempts to manipulate participation in QS surveys and therefore results," it dropped the option for people to sign up to participate in surveys. But QS said it believes that "the sign-up facility represented a crucial source of respondents and the process enabled thousands of interested academics or employers to actively signal their interest in participation." So it restored the sign-up option last year, but to "ensure that institutions are not using the sign-up process to unduly influence the position of their own or rival institutions, we have introduced a question which requires respondents to indicate from where/which institution they have heard about the facility."
Phil Baty, who leads the rankings effort at Times Higher Education, said via email that the Trinity letters demonstrate a continuing problem with allowing people to volunteer for rankings. "Times Higher Education has always employed an invitation-only reputation survey," he said. "We select senior, published academics at random but we distribute the survey to ensure a fair and accurate geographical and subject balance, based on the true distribution of scholarship across the world according to United Nations data. We do not, and have never, allowed anybody to sign up to take part in the survey and we do not allow universities to nominate any academics or employers to take part. These are absolutely fundamental and very simple rules to protect the integrity of the survey and to ensure that it is statistically robust."
Bizzozero rejected the idea that its surveys are vulnerable. "QS runs sophisticated screening analysis to detect anomalous patterns in response and routinely discards invalid responses," she said. "Any attempt to manipulate the results or to solicit responses may result in the disqualification of all responses for that survey for that year, invalid or otherwise, where the source cannot be verified as entirely independent."
She also said Times Higher has any vulnerability that QS faces. "Whilst THE have an 'invitation-only' approach to surveys, it is probable that some of the respondents intersect with those touched by the misguided TCD campaign. A lower overall sample size, and the practice of drawing outcomes from a single year of response, will render their exercise more statistically sensitive," she said.
