Inflated Assessment by Phoenix?

In a new, first annual report on academic outcomes, the nation's largest private university -- and the largest for-profit -- makes a case for value-added education. The University of Phoenix reports that on the Educational Testing Service’s Measure of Academic Proficiency and Progress (MAPP) test, “University of Phoenix seniors score significantly higher than their University of Phoenix freshmen counterparts in all areas."

The report emphasizes comparisons of Phoenix's (lower) freshman and (higher) senior year scores in critical thinking, reading, writing, mathematics, humanities, social sciences and natural sciences. “Results show that as a group, University of Phoenix students make significant progress in basic content areas from freshman to senior levels. University of Phoenix students often enter their studies with lower scores in the general education areas as compared to more exclusive institutions but perform at levels comparable to seniors at other institutions by the time they graduate.” In reading, for instance, Phoenix freshmen, defined as students with one to 30 credits, attain average MAPP scores of 116.45, compared to 117.20 for freshmen from all institutions, whereas Phoenix seniors have MAPP scores of 119.27, just below the 119.72 average for all students.

But are Phoenix freshmen and seniors actually “counterparts,” as the report claims? The university's public relations office indicated Thursday that it did not have adequate time to respond to a question on the proportion of seniors who transferred at or above the sophomore level, but the report itself states that the student who starts and finishes a degree at the same institution “is an anomaly at University of Phoenix." For the university's 165,000+-student online campus, its federally-reported graduation rate, reflecting only the number of full-time, first-time entering college students who complete a bachelor's degree within six years, is just 5 percent.

Given its high transfer rates of mid-career professionals with some college education and significant workplace experience -- the university began as a "degree-completion institution," where students "sought out to finish what they had started" -- how does Phoenix know the disparities in freshman and senior year scores are attributable to the Phoenix education and not reflective of the different demographics of those who fall under "freshman" or "senior" status in terms of credit attainment? The university only recently began catering specifically to first-time college students through its two-year degree granting college, Axia, where it now steers students who come in with few or no transfer credits.

“These are really different populations. I would tend to think that that would be the disparity between their different skill levels, frankly," said Trace A. Urdan, an analyst with Signal Hill, an investment group.

“It gives kind of a false impression to suggest that they should somehow be credited with the improvement," Urdan continued relative to Phoenix's report. “I’m not saying that they’re not helping these students and it may in fact be that the first-time students who are starting in Axia, it may be after two years, you would see gains on that same test. But that’s not what they presented in the report. What they presented in the report was PR, and not very clever."

Phoenix's leader, however, described the problem of tying measurable gains to classroom learning as a universal one in higher education. “How do we know that people who come into the Ivy League and leave who are highly successful can attribute that directly to the education they receive there? That’s always a bit of a dicey proposition for all of higher education,” Phoenix’s president, William J. Pepicello, said in an interview Thursday. He added that the university did not break down scores by transfer status for this first report but that he fully expects that from it, future research questions will emerge.

“The point we were trying to make in the report is if you look at where students who graduate from the University of Phoenix are compared to the rest of higher education, we’re very comparable. There’s always a mix of reasons how people get there," Pepicello said.

“We hope that a major part of that is linked to their experience at the university with the curriculum.”

In releasing its first Academic Annual Report, Phoenix invokes the language of the Secretary of Education's Commission on the Future of Higher Education, citing “access, accountability, quality, and affordability.”

“Lots of people think they know about the University of Phoenix and they think they know what we do and who we are, but for the most part people tend to focus on the business side of what we do,” said Pepicello.

“One of the things I wanted to do was tell the academic story of the University of Phoenix and talk about how we are addressing the social agenda of the university.”

The report, for instance, emphasizes the university's diverse student body – just 54.3 percent white, compared with 61 percent nationally, with more than twice the percentage of African Americans enrolled in 2008 than the 2005 national average (24.6 versus 12 percent).

In addition to the MAPP, the report includes comparisons on the Standardized Assessment of Information Literacy Skills (SAILS), which 1,170 Phoenix students took. The university found that Phoenix students performed significantly better than students at all participating SAILS colleges in searching, evaluating sources, and understanding economic, legal and social issues, while at statistically equivalent levels in developing a research strategy, selecting finding tools, using finding tools features and retrieving sources. Phoenix students scored significantly below others in documenting sources.

Also in the report, the university argues that when all students are counted -- as opposed to only first-time students, as allowed under federal graduation rate formulas -- degree completion rates are comparable with national averages (38 percent for bachelor degrees versus 43 percent nationally, for instance, and 27 percent for associate degrees, equal with national norms). The university also argues that Phoenix students with risk factors inhibiting graduation – like enrolling part-time, being a single parent, or working full-time while enrolled – graduate at higher rates than peers with the same number of risk factors nationally. In its report, the university stresses its mission of serving high-risk students. "Inclusion and reaching out to underserved populations are at the heart of the university's mission," the document states.

“They’re laying out a lot of things about who they are and how they work," said Clifford Adelman, senior associate for the Institute for Higher Education Policy. "They’re trying to do comparative analysis; this is what all schools are falling over each other to do.” Adelman added that critiques relative to freshman to senior year growth apply not just to Phoenix, but to any number of colleges attempting to prove their "value-added" status in similar ways through other generalized, standardized testing measurements.

“I’m giving them credit for trying to do something. Everyone’s trying to do something and their motivations are fine. But what they’re trying to do does not really reflect their purpose for being an institution.”