Ranking the Online Colleges

U.S. News & World Report, which in the last three decades has become one of the most successful and controversial kingmakers in higher education, is taking preliminary steps to apply its rankings to the increasingly important realm of online colleges.

Meanwhile, a U.S. News spin-off site accidentally profiled a new program at an online institution considered by some authorities to be a "degree mill" — a mistake that, while it was swiftly corrected when brought to the attention of the editors, highlighted just how difficult it can be to size up online institutions in the current environment.

Brian Kelly, editor of U.S. News, this week sent out a memo to more than 1,000 officials at online colleges and university-based online programs notifying them that, over the next few weeks, the publication would be soliciting data from their institutions with an eye to ranking programs that are delivered at least 80 percent online. The target release of the inaugural U.S. News online program rankings is mid-October.

“With the rapid growth of online programs in higher education, prospective students are asking for more, and more useful, data to make informed choices,” Kelly wrote in the memo. “We are creating a site that will bring the same quality of information to online consumers, and the same opportunity for schools to connect with those students, that we've brought to brick and mortar institutions over the last three decades.”

U.S. News is taking aim at just a few types of online programs this time around: master’s programs in business, computer information technology, education, engineering, and nursing; and baccalaureate programs. (These are the programs with the highest online enrollments, according to U.S. News.) The publication will not be sizing up any online associate or doctoral degree programs, nor will it be ranking institutions overall, as it does with traditional colleges and universities.

The formulas for these rankings have not been determined. But the online versions could differ from the traditional rankings in several ways, said U.S. News officials. For example, the initial rankings will not include peer-assessment scores, in which college presidents rank institutions that are similar to their own -- historically one of the publication’s most controversial metrics.

The rankings arrive at a time when typing “best online colleges” into an Internet search engine is more likely to bring a potential student to a lead-generation website — a site that collects their contact information and educational interests and sells it to recruiters — than any kind of rigorous, data-driven assessor of various online options.

U.S. News hopes to provide a tool for evaluating online programs based on "old-fashioned" data collection and analysis, said Robert Morse, the director of the rankings.

In interviews on Wednesday, Morse did not want to talk about specific rankings methodologies because they have not yet come up with criteria for assessing the different types of online programs — and also because they do not want respondents to withhold certain data because they think it might result in a bad ranking. The plan is to solicit a wide range of data, and then decide on criteria based on a combination of what makes sense, according to scholarly research into online course effectiveness and interviews with online education authorities, and what comes back, said Morse.

However, the rankings director did give some indications of what data U.S. News is seeking. For example, there are questions about the degree to which faculty members are trained to teach online; whether the same faculty members teach the online version of a course that teach the traditional classroom version; what proportion of faculty are adjuncts; the extent to which a program polices cheating on online tests; how much debt the average student takes on and job placement and salary upon graduation (it will not be asking about program-level loan default rates); and a number of traditional metrics, such as graduation and retention rates.

Both Kelly and Morse acknowledged that one of the biggest challenges of compiling the rankings will be getting cooperation from for-profit colleges, which make up a significant part of the online sector but generally shy away from giving up data they are not required by law to disclose.

But many traditional institutions were no different when U.S. News first began soliciting them for rankings data in 1983, said Kelly. Eventually, many “realized it was in their interest, and it became national standard,” he said. He said he hopes proprietary online institutions will arrive at the same conclusion. “Our feeling is the good institutions will want to share these data,” Kelly said. “And that we’re going to work with them to make sure we get accurate info in people’s hands.”

Morse emphasized that the initial survey and methodologies will not be perfect. “Any ranking or evaluative list that we do is going to be our first attempt,” he said, “and we know as data get better they’ll evolve over time to become more robust and sophisticated.”

The upside is especially high with online institutions, said Kelly, since they have more data on student outcomes than do traditional colleges and universities. With the amount of data programs are collecting through their online learning environments, U.S. News believes it can not only match the reliability of its current rankings with the online version, but exceed it. Online programs “are about data and measurement,” said Kelly. “And when you have great data and measurement ability you can create great rankings.”

Generation Gap

At the same time that U.S. News was promoting its expansion into online college ranking, a spin-off site it opened two years ago unwittingly wrote a plug for a new program at an online institution, Almeda University, that is not recognized as a legitimate degree-granting university by the U.S. Department of Education or any mainstream accrediting agency, and which has been flagged as a “degree mill” by the Oregon state government.

“Working adults who want to pursue a psychology master’s degree can benefit from the flexibility of online programs, such as the one offered by Almeda University’s School of Psychology,” said a news brief posted Monday on U.S. News University Directory.

The editors of the site took down the posting after it was brought to their attention by Inside Higher Ed, which was tipped off by the watchdog site GetEducated.com. “As a matter of principle we generally focus on accredited colleges and universities and the programs they offer,” the University Directory editors wrote in an e-mail. “On the rare occasion, such as this, when we learn that the institution’s accreditation is in question, we take appropriate action.”

The publication opened University Directory two years ago in bid to move into the business of lead generation through a partnership with Bisk Education, Inc.

The move raised eyebrows among some critics, who wondered whether U.S. News’ attempts to capitalize on its authority in the higher education world might in fact compromise its credibility as a disinterested observer of the industry.

Kelly said the news brief about Almeda’s psychology program was the result of simple editorial oversight, not any kind of advertising deal with Almeda.

Press releases might be parroted in the pages of University Directory, not always with attribution, said Kelly; not because universities are paying for the exposure, but because such releases are just “part of the news flow” in the content-obsessed world of the modern news industry. Another effect of the push for more content is that errors are more common than they were before — at U.S. News as elsewhere, Kelly said.

Indeed, the gaffe suggests that the opportunities and demands of the Internet age complicate matters both for higher education institutions and those who cover them.

“As our model has changed to a digital model, we are publishing so much more content in so many different areas,” said Kelly. “The demand for the information we publish is enormous — it’s bottomless. And we are continuing to experiment and monitor the best ways to produce that information.”

For the latest technology news and opinion from Inside Higher Ed, follow @IHEtech on Twitter.