A Rankings Mess, Getting Worse

Were four years of top ratings from U.S. News based on incorrect data?

January 29, 2018
 

U.S. News & World Report last week announced that it was removing the ranking of Temple University's online M.B.A. program from its 2018 Best Online Programs list. In the days since, more questions have been raised about the inaccurate information Temple apparently provided U.S. News and the potential for such incorrect information to influence several years of rankings.

The magazine's announcement was straightforward. Part of the formula used by U.S. News is based on standardized test scores, and score averages carry less weight when under 75 percent of new students submit scores. In the case of Temple's online M.B.A. program, the business school originally reported that all of its students submitted test scores. In fact, only 20 percent had done so.

Having 100 percent of students take standardized tests -- for business schools, either the GMAT or the GRE -- isn't implausible. It's what would happen if the Temple business school required the tests of applicants. (Temple does so for its in-person M.B.A. program but not for its online program.) And U.S. News noted in its announcement that it expects business schools (and others) to submit accurate information.

"U.S. News relies on schools to accurately report their data. In this case, the Fox School of Business submitted the data on its own during the summer and fall 2017 data collection period and also completed the data verification process, assuring U.S. News that the data were accurate."

U.S. News and other entities that conduct rankings periodically find out about incorrect data and, sometimes, adjust rankings as a result. In this case U.S. News declared that Temple would be "unranked" until next year's rankings come out.

What U.S. News didn't state was that the position from which Temple's online M.B.A. was being bumped was the top one. Nor does U.S. News note that this was a position held by Temple for several years, in all of which it was reporting the same data that now have been discredited.

Image of news release, dated Jan. 9, 2018, from Temple University with headline “Fox Online MBA Ranked No. 1 Four Consecutive Years by U.S. News & World Report,” by Christopher A. Vito. Article begins, “Temple University’s Fox School of Business remains the nation’s leader in online MBA and online undergraduate business programs. For the fourth consecutive year, U.S. News & World Report has ranked the Fox School’s Online MBA program No. 1 in the nation, according to the publication’s 2018 rankings of the best online programs.”Indeed earlier this month, Temple issued a news release on its ranking. The release has since been removed, but a cached version (at right) is still available and features numerous quotes about how strong the online M.B.A. program is.

After the U.S. News announcement, the website Poets & Quants, which focuses on business schools, started digging around in the data Temple had reported to U.S. News for the previous (No. 1) rankings.

It turned out that in the years 2014, 2015 and 2016 (the three years prior to the most recent for which Temple was ranked best online M.B.A. program), the university had also reported that 100 percent of its students had taken standardized admissions tests. In the two years prior to that (when Temple hadn't been the top program), the percentages were 25 and 33 percent.

As Poets and Quants wrote, it would seem highly unlikely that an M.B.A. program would have no more than a third of its students taking standardized tests for several years, and then jump to 100 percent for several years straight, and then drop down to 20 percent, the percentage Temple now concedes is correct for the 2017 entering class.

The U.S. News announcement last week only covered the most recent rankings, and the magazine did not respond to questions about what -- if anything -- it would do about the potential that several years of rankings were based on incorrect data, except to say that it would review whatever information Temple provides.

A spokesman for Temple said that the university's president, Richard M. Englert, on Thursday decided that the university would hire an outside reviewer to examine what happened with the data. While the reviewer has not been hired yet, the spokesman said the review would include previous years, not just the year that U.S. News found was incorrect.

"The results of that review will go to the president, and he will take appropriate action based on those results," the spokesman said.

Read more by

Be the first to know.
Get our free daily newsletter.

 

 
+ -

Expand commentsHide comments  —   Join the conversation!

Today’s News from Inside Higher Ed

Inside Higher Ed’s Quick Takes

What Others Are Reading

  • Viewed
  • Past:
  • Day
  • Week
  • Month
  • Year
Back to Top