(Not Really) Measuring Up

There are bright spots in Measuring Up 2008, the biennial "report card" on higher education that the National Center for Public Policy and Higher Education is releasing today. The proportion of students graduating from high school prepared to do college-level work is on the rise, for instance, and the percentage of high school freshmen who go on to enroll in college is also climbing. Even the rate at which those who enroll in college go on to get a degree is edging up.

But as has become a tradition for the report and the center's director, Patrick M. Callan, a longtime analyst who has long taken a bearish view of higher education, the positive developments in the latest iteration of Measuring Up are overwhelmed by the "buts" (small gains in college preparation are mitigated by drops in high school graduation, increased enrollment among traditional college-age students offset by declines in enrollment of adult students) and, Callan and the report argue, by the increasingly global context in which American higher education's performance must be viewed.

Taken together, the performance of colleges, states and, ultimately, the country in educating its citizens is inadequate to the current needs. “In most of our categories, we see some very modest progress, but the progress does not begin to match the magnitude of the challenge we face, or the progress other countries are making,” Callan said during a news briefing about the release of the report.

The report’s results would be troubling enough, Callan suggested, without the specter of the country's current economic turmoil. But with it -- given that troubled financial times historically lead to reductions in state and federal support for higher education -- the implications of the report’s findings are even more serious, he said.

“Our previous reports have been significant, but this one comes out at a very significant time,” said Callan. “This is not a propitious time to have a level of higher education decline. If we respond to this recession, when the potential students we most need to get into the system are those who can least afford it, in the way we usually do” -- by raising tuitions -- “we will set ourselves back quite a ways.”

Rating the Results

Measuring Up has been controversial since its inception in 2000, seen by some critics as overly alarmist and unfairly critical. (Cliff Adelman, a longtime Education Department researcher and now an associate at the Institute for Higher Education Policy, calls the series "an annual ritual, complete with its own liturgy of grey noddings and self-flagellation." More from Cliff later.)

And indeed, Callan and his national center have at times aligned themselves with aggressive critics of higher education; Callan informally advised Charles Miller, who headed Education Secretary Margaret Spellings’ Commission on the Future of Higher Education, which argued that American higher education’s best days were behind it unless it could reinvent itself.

Callan shares that view to some extent; at the news briefing, he described as an “American myth” the notion that just because the country is home to many of the world’s leading universities, that automatically means that its higher education system as a whole is the world’s best. “If you measure higher ed that way, we look pretty good,” he said. “But if you say a good higher education system is one that meets at least international standards” on things like college completion, “then we’re not leaders any more.”

But writing Measuring Up off as the work of cranks -- or at least of cranks without power, potentially -- would be a mistake. The foreword of the report was written by James B. Hunt Jr., the former North Carolina governor and chairman of the national center's board, whose name has been bandied about as a potential candidate for education secretary in the Obama administration. And its advisory board is an august body of researchers and policy makers (Gordon Davies, David Breneman, Emerson Elliott) well-respected for their knowledge of and advocacy for higher education, even if many of them (Peter Ewell, Jane Wellman, Margaret A. Miller) are seen as being hawks rather than doves on the need for colleges to change the way they operate.

The basic goal of the Measuring Up report (this is the fifth biennial report) hasn't changed: to evaluate "the progress of the nation and all 50 states in providing Americans with education and training beyond high school through the bachelor's degree." In that way, it focuses not on the overall health and status of American higher education as some might define it -- including the research enterprise -- but squarely on how successful colleges are at educating undergraduates, primarily from a quantitative rather than qualitative standpoint.

This focus represents the emerging consensus -- embraced not only by the Spellings Commission but by numerous other reports and groups -- that the biggest challenge facing American higher education is the need to ratchet up the number of educated young people and adults to replace the masses of baby boomers who will be retiring (perhaps a little later than they might previously have been, given the shrinkage in retirement savings) in the coming decade. This task is made more difficult by the fact that the fastest-growing segments of young people coming into the educational pipeline are from groups -- such as underrepresented minorities and low-income families -- that have historically been least prepared for college and least well-served by the higher education enterprise.

In Measuring Up, states are evaluated and graded on six basic categories -- preparation for college, participation (opportunities for education and training beyond high school), affordability, completion (persistence in and completion of certificate and degree programs), benefits (contributions by college-educated and trained residents to their states' economic and civic well-being) and learning (college-educated residents' performance on a variety of measures of knowledge and skill).

States receive grades based on their relative performance against one another, benchmarked against those that perform best each year. The report also includes indicators that show states' performance over time.

A Report Card Not to Take Home to Mom

The states performed best on preparation and completion, worst on affordability (49 F's) and learning (all incompletes). Highlights are below:

• Preparation: 6 A's (Colorado, Connecticut, Maryland, Massachusetts, New Jersey, Vermont), 18 B's, 21 C's, 5 D's, and no F's. Thirty-four states showed improvement or stayed the same on the number of 18- to 24-year-olds with a high school credential, but the high school graduation rates of black and Hispanic students in many states lagged badly (82 percent of black young adults in Illinois had a high school credential compared to 95 percent of their white peers; 56 percent of Hispanic 18-24-year olds in North Carolina had a high school degree, compared to 92 percent of whites.)
• Participation: 2 A's (Arizona and Iowa), 8 B's, 22 C's, 15 D's, and three F's (Alaska, Louisiana, and Nevada). Forty-three states improved or stayed the same on the number of 18- to 24-year-olds enrolled in college, but a majority of states showed decreases in the number of 25- to 49-year-olds in college-level education or training. On this and other measures, the gaps by racial and socioeconomic status are significant.
• Affordability: 49 F's and one C grade, for California. "The whole country has gone south on affordability," said Callan. He called the picture a "national disaster" as tuition continues to outpace family income, increasing the burden of paying for college particularly for low- and middle-income families. The states are graded on families' ability to pay (percentage of income needed to cover the students' costs minus financial aid) at different types of institutions, the states' emphasis on need-based aid (their own investment in such aid as a percentage of the federal investment in their states' students) and lower-cost colleges, and students' reliance on loans. Two states improved or stayed the same on the percentage of family income needed to pay for a four-year public college, while 48 states fell on that measure.
• Completion: 11 A's ( Iowa, Massachusetts, Minnesota, New Hampshire, North Dakota, Pennsylvania, Rhode Island, Vermont, Washington, Wisconsin, Wyoming), 20 B's, 16 C's, one D and two F's (Alaska and Nevada). All but those two states improved in the number of college degree completions per 100 students, but the caveats here, in the eyes of the Measuring Up crew, are that the rates remain low, especially as measured against those in other countries. "The United States' world leadership in college access has eroded steadily," he wrote in an analysis of the report. "In college completion, which has never been a strength of American higher education, the U.S. ranks 15th among 29 countries compared." While older Americans still fare well in international comparisons of degree holders, "the U.S. population has slipped to 10th in the percentage [of 25- to 34-year-olds] who have an associate degree or higher. This relative erosion of our national 'educational capital' reflects the lack of significant improvement in the rates of college participation and completion in recent years."
• Learning: Here's what the report had to say on this front: "All states receive an 'incomplete' in learning because there are not sufficient data to allow meaningful state-by-state comparisons," a point made in by Ewell in an Inside Higher Ed essay last month.

An undercurrent of the report is the significant gap that exists between the haves and the have-nots on college access and affordability. "It has always been an ethical and moral problem that we undereducate minorities and low-income students," Callan said in an interview. "But for the first time, we are going to pay an economic price as well" if more of those Americans are not made ready to enter the work force and contribute to society.

Of rapidly rising tuitions, he added: "I don't know anybody who would argue that we can do this for another 25 years and remain accessible."

A Critical Voice

Cliff Adelman has generalized criticisms of Measuring Up as well as specific complaints about the latest iteration. Generally, he said in an e-mail message, the series "never misses an opportunity to tell states how badly they are doing, and it sure keeps state media outlets happy with copy and occasions to groan. No grade inflation in this game: out of 250 grades delivered on state performance by the authorities of “Measuring Up,” only 82 are B- or better (to be sure, there are 49 Fs passed out on the “affordability” criterion, but that’s something that, in the middle of a major recession, your grandma could have told you)."

Adelman takes particular issue with the report's dependence on the Organization for Economic Co-operation and Development's "population ratios" to compare other countries' performance with that of the United States. "OECD has used them to bypass a host of inconsistencies in the ways its 30 member countries report education data, but the 30 member countries also have different census methodologies, so the components of the denominator from Sweden are not identical with the components of the denominator from Portugal," he writes.

"Why is this important? When denominators are flat or declining and numerators remain stable or rise slightly, percentages rise; and vice versa when denominators rise faster than numerators.... If you use population ratios, and include the U.S., it’s going to look like we’re 'declining' -- which is Measuring Up’s preferred story." That is especially true now that half of European students are pursuing three-year degrees because of the Bologna process, Adelman notes -- something "I'm stunned that Measuring Up didn't mention at all."

Adelman concluded his critique by saying he hopes that while the "Bush Administration’s Spellings Commission elevated 'Measuring Up' as its own Bible, one can only hope the Obama administration will exhibit a healthy dose of skepticism, and look instead to more constructive analyses that will make a difference in the lives of students."

While much of Measuring Up is dedicated to showing the extent of the problems in colleges and states as Callan and his colleagues see them, it offers a conclusion of its own that very much aims to represent the interests of students -- particularly when, as recent headlines suggest, states are preparing to respond to the current economic crisis in their long-established ways.

"They can respond to their current budget crises in the usual patterns of the past, by allowing tuition and student aid policy to play second fiddle to institutional finance," the report states. "States that select this course will most likely see precipitous tuition increases, cuts in student financial aid, and drops in college access," widening the existing gaps in access and completion and making college less affordable.

"But states have another option: to establish state policies for tuition and student aid that balance the financial burden for higher education among states, the institutions of higher education, and students and families. This is both a short- and long-term strategy that makes state policy more transparent, grounds it in the needs and financial circumstances of state residents, establishes college affordability as a priority, protects educational opportunity, and in the process helps to meet the needs of states and the nation for a well-educated work force and citizenry."