U.S. News & World Report’s rankings of the best online programs are growing more robust with each iteration, its critics acknowledge, but the publication is exaggerating by calling them the “only resource students can turn to for unbiased information on online programs.”

The fourth edition of the rankings, which were introduced in 2012, went live this morning -- the latest attempt by the publication to provide an overview of the rapidly expanding number of institutions that offer distance education programs. In three years, the number of ranked programs has nearly doubled, from 677 in 2012 to more than 1,200 this year.

Over the course of those years, the formula behind the rankings has also changed. Last year, U.S. News introduced a “peer reputation” factor to the mix, which accounts for 20 percent of a bachelor’s degree program’s score. The other factors -- including admissions selectivity, faculty credentials and training, student engagement, student services and technology  -- each had to shed 5 to 10 percent of their weighting as a result.

That addition was a controversial one, particularly to rural or younger institutions. With a large share of the score being awarded based on the opinions of others, some institutions say the system rewards colleges and universities based on name recognition, not the quality of education.

This year’s rankings feature mostly minor weighting tweaks, suggesting U.S. News is approaching a formula it finds ideal. The publication’s data collection process, however, still gives pause to some critics.

“These rankings have improved greatly over the last few years as U.S. News has been open to input,” Russell Poulin, deputy director of research and analysis for the WICHE Cooperative for Educational Technologies, said in an email. But he added that “The rankings have a long way to go.”

U.S. News depends on colleges and universities to self-report data. To rank online bachelor’s degree programs, for example, the publication conducts a two-stage survey. First, it asks regionally accredited colleges and universities if they plan to offer such programs. If yes, the publication follows up with a questionnaire to collect data on graduation and retention rates, student debt and faculty development efforts, among other factors. This year, 297 colleges and universities said they offer online bachelor’s degree programs, up from 283 last year.

As with other self-reporting surveys, some institutions choose not to participate. Others may be incorrectly ranked because of reporting errors -- or outright fabrication. The questionnaires also asks colleges and university to include data on factors such as class sizes; recent research has questioned whether or not they actually have an impact on student outcomes.

Institutions with multiple programs are instructed to aggregate their data to “reflect the most typical experiences” for students and faculty, which Poulin described as the “basic, fundamental flaw” of the rankings.

“If data are aggregated across programs within a college, how does the student know about the differences in individual programs within the institution?” Poulin wrote.

U.S. News has previously acknowledged the various criticisms, saying the rankings are meant to help students find programs suitable for them.

“As with all our rankings, our goal is to provide accurate information that allows consumers to make apples-to-apples comparisons,” editor Brian Kelly wrote in the introduction to last year’s rankings. “We don't pick favorites -- the numbers do. But we also stress that numerical lists might not be the right way for some people to decide. The point is to find what works best for you.”

In the press push leading up to Wednesday’s release, the publication was less modest. A spokeswoman for U.S. News described the rankings as “the only resource students can turn to for unbiased information on online programs,” saying, “There is currently no accrediting body for online programs.”

That statement ignores national accreditors such as the Distance Education Accrediting Commission, said Ronald Legon, executive director of Quality Matters. DEAC last week changed its name to more clearly signal that it does in fact accredit distance education providers. The organization was previously known as the Distance Education and Training Council.

“More importantly, the six regional accrediting agencies pay increasing attention to the online component of the institutions they accredit,” Legon said in an email. “But the U.S. News claim is misleading in another respect. Accrediting agencies do not rank the quality of the programs they accredit from best to worst, whether for all programs, face-to-face programs or online programs. These agencies judge whether institutions and programs meet their published standards.”

By only surveying regionally accredited institutions, U.S. News “leaves out a lot of the innovations,” said Leah K. Matthews, executive director of DEAC. She pointed out that institutions such as Western Governors University and American Military University were both accredited by DEAC before receiving regional accreditation.

Legon, whose organization conducts quality assurance in online courses, also took issue with the other part of U.S. News’s claim -- that the rankings are an attempt to “put some standards around online education where there are none,” as Robert Morse, chief data strategist at U.S. News, said in a press release.

“In fact, online education is and has been more highly scrutinized than face-to-face education, the presumed benchmark of quality,” Legon wrote. “Standards have been promulgated for the design and delivery of online education by the recognized accrediting bodies and a number of associations and specialized organizations, Quality Matters among them.”