Challenging the Measures of Success

Could Cal State-Long Beach be educating students better than Princeton? It depends on what you think of graduation rates.
June 6, 2007

F. King Alexander, president of California State University at Long Beach, wants to change the way people evaluate what a college contributes. "I like to ask people: Do you want Princeton or Cal State Long Beach in your economy?"

To those who live by U.S. News rankings, or SAT scores, or prestige, or Nobel Prizes, or graduation rates, the answer is a no brainer: Princeton. But to Alexander, there's a simple way to change the equation. Instead of thinking about graduation rates, which are an easy proxy for SAT scores, competitiveness, and all kinds of other factors that relate to the wealth or prestige of an institution, he wants people to think about how many students graduated. In other words, focus on the raw numbers, not what percentage met the federal definition for graduating.

"We will have more graduates this year than Princeton has students," Alexander said. (Long Beach graduates more than 8,000 students a year, while Princeton's total enrollment is about 6,700.) "And we're going to have 500 engineers who graduate this year, and 300 nurses, and 1,100 school teachers and they are all getting good strong degrees and are getting very good jobs."

In contrast, when you look at graduation rates, Princeton comes out on top, with a rate of 97 percent, compared to 48 percent at Long Beach, using the federal definition, which looks at first-time, full time enrollees who earn degrees within six years (or three years for a community college).

While such rates mean something to many people, Alexander said that they actually reflect a specific set of incentives, which even if appropriate for Princeton aren't appropriate for most places. "If you focus on a rate, you drive public universities away from their public missions. Everyone knows that to get your graduation rate up, the best way to do that is turn away all the academically challenged students and there is evidence of this all over the United States."

As a result of his views, Alexander is working with officials of the American Association of State Colleges and Universities to try to change federal policy and national perceptions about graduation rates and whether they are a good measure. While the project is still in the idea stage, it comes at a time that other groups are also considering proposals to change the way graduation rates are calculated. And while the federal definition has long frustrated some educators, there appears to be more discussion now about seeking change than has been the case previously.

Alexander's ideas is to shift away from a single measure of graduation rates and to use a three-pronged approach: total graduates, percentage of students who are eligible for Pell Grants (a proxy for serving disadvantaged students), and the traditional measure of graduation rates.

Needless to say, many college presidents like to look for statistics that portray their institutions in favorable ways. Alexander insists that what he's talking about would raise the rigor with which all colleges are evaluated. By keeping the traditional measure as one tool, he said, colleges that were graduating hardly anyone would get scrutiny. But it is by combining the measures that a full picture would emerge, he said. If two colleges both have the same graduation rate, but one is graduating many more students and with a larger share of Pell-eligible students, that college is performing better and should be recognized as such.

Generally, he said, the emphasis on graduation rates encourages colleges to make admissions competitive. While institutions like his would benefit from the comparison Alexander is proposing, he is quick to note that others would as well. He cited three private universities, Brigham Young and Cornell Universities and the University of Southern California, as institutions that may be slightly less competitive to get into than their various peers, but that graduate far more students -- and from economic bases that include larger shares of Pell eligible students than their peers. Why should they be viewed as having lower quality, he asked?

"Currently all of the pressure flows in one way -- to do a good job with the best prepared students," he said. But given demographic trends -- with more of those needing a college education coming from high schools where they may not be prepared -- that should change, Alexander said. "If we continue on the path we are on, then we are all going to be competing for out of state women, who score better, finish faster, and will lift everyone's graduation rates. Is that a good societal direction?"

Edward M. Elmendorf, senior vice president for government relations at AASCU, said "what we're trying to do is to find a different template, a different way of showing information."

Elmendorf also stressed that Alexander's concept would not only benefit institutions like Long Beach. Many historically black colleges, Elmendorf said, pride themselves on admitting students who graduate from poor high schools and who may lack the resources or academic preparation to progress speedily to graduation. Not surprisingly, graduation rates at many of these institutions are lower than they are at institutions that admit wealthier, better prepared students.

By adding in the kinds of factors Alexander is talking about, Elmendorf said, these colleges look very different. They are among the highest in educating Pell-eligible students and they are graduating plenty of students, even if not on a schedule associated with elite higher education.

There is interest in seeking amendments to the Higher Education Act to change the graduation rate formulas, Elmendorf said. And he added that these rates are influential beyond that law, but are relied on by state lawmakers, journalists and others looking for a quick read on an institution -- a read that Elmendorf said is currently inaccurate.

Community colleges have long been frustrated by the federal formula, which generally fails to reflect the large proportion of their students who enroll part time, enroll at multiple institutions or enroll with no intention of completing a degree. David Baime, vice president for government relations at the American Association of Community Colleges, said that while his organization hasn't been pushing a specific proposal on the issue, he was pleased to see interest in changing the way people think about graduation rates.

"Our students have so many different goals, and for many of our students, their ultimate goal is not to graduate but to improve themselves economically through getting a job," Baime said. "All of these considerations make graduation rates for our sector very different to compare. I think that looking at a simple number across sectors conceals more than it reveals."

Another approach being discussed by some is the idea of replacing the graduation rate with a "success rate." The National Association of State Universities and Land Grant Colleges -- together with AASCU -- is working on a new voluntary accountability system that provide new ways of evaluating college quality. One idea being considered is the success rate approach.

David E. Shulenburger, vice president of the land grant group and formerly provost of the University of Kansas, said that the current graduation rate data don't reflect the way students transfer or take longer than is considered appropriate to finish their degrees. He provided an example of an (unidentified) university involved in these discussions. Four years after enrollment, only 28 percent of students have graduated. But 5 percent have graduated from another institution, 38 percent were still enrolled at the institution, and another 14 percent were enrolled at other institutions. Counting all of those, that's an 85 percent success rate -- very different from a 28 percent graduation rate.

Shulenburger would not go as far as Alexander and thinks there needs to be attention paid to those who don't graduate or transfer or continue. When he was at Kansas, he said, "I noticed every year that we took in about 4,000 students and graduated about 4,000 students with bachelor's degrees and we didn't have a 100 percent graduation rate."

But Shulenburger said that the current definition of graduation rates assumes that students and their parents care only about one model: where a student enrolls at one institution, stays there and graduates as soon as possible. "Students are different," he said, and that model will work for some and not others. An institution that provides a solid education and sends many on to other institutions to graduate is doing well, he said.

And he agreed that the traditional measure sends the wrong message to the many institutions whose mission is not to be as elite as possible. "The U.S. News kind of approach has been pretty destructive," he said. "You certainly can change your graduation rate by changing the type of students you admit," he said. "There's lots of evidence of that."

Some experts on graduation rates fear that some of the ideas being discussed may not focus enough attention on what happens to individual students who enter a given college. Clifford Adelman, a long time Education Department researcher who is now a senior associate at the Institute for Higher Education Policy, agrees that there are problems with the current system. He has proposed changes that would better account for the reality that traditional and non-traditional students have different goals and pressures, and that transfer is not necessarily a bad thing.

But Adelman believes that the rates can be made meaningful and he is very skeptical of a move away from the concept of rates. Too many institutions, he said, "will admit any student with a warm body and a test score." Colleges need to be serious about working with those they admit, and to be accountable for those they admit, Adelman said, and graduation rates -- with some tweaks -- could do that.

Adelman said that an institution may be healthy if it is graduating large numbers of students. But what of the students who don't graduate? The proposal to abandon rates, he said, "is at best misguided" and is "very typical of institutional leaders to focus on institutions and not students."

Others are more intrigued by the approach, even while seeing the difficulty of pulling off a change.

Travis Reindl is running the Making Opportunity Affordable Project for Jobs for the Future. He recently released a study arguing that the United States faces a series degree shortage and needs to increase degree production by 37 percent by 2025 to keep up with international competitors and economic and social needs.

To reach those goals, he said that American higher education will need to rely on institutions that reach out to students who aren't admitted to elite colleges. "We have to look at metrics that account for the whole sweep of higher education, and not only those that benefit one group over another."

While the graduation rate "is still a relevant indicator," it should be viewed "in combination with other factors," he said.

Still, Reindl warned that change won't be easy. "Policy makers have really latched on to the graduation rate because it's easy to understand," he said. "You are going to have to do some serious education so people understand the changes and see a new valid measure and not some way to give institutions a pass."


Be the first to know.
Get our free daily newsletter.


Back to Top