Cost-Effectiveness, or Cost?

Friday’s IHE did a story featuring a report by Douglas Harris and Sara Goldrick-Rab that’s well worth reading in its entirety. In a nutshell, it measures the ‘productivity’ of various programs, using what boils down to dollars-per-graduate. Among other things, it suggests that call centers to nudge students into attending class have great bang for the buck, but that Upward Bound and similar programs are wildly expensive for what they achieve.

The goal of the study -- which is entirely to the good -- is to encourage colleges to base resource allocation decisions on actual effectiveness, rather than on what sounds good or what has usually been done. The authors break out two-year and four-year sectors -- thank you -- and actually define their variables. (Notably, the productivity decline over the past forty years has been far more dramatic in the four-year sector than in the two-year sector.) Even better, they acknowledge that most of the research done on various programs are done on those programs in isolation, rather than in comparison with each other. If we’re serious about dealing with limited resources, we have to acknowledge that money spent on program A is money not available to be spent on program B. It’s not enough to show that a given program helps; it needs to help more than its alternatives would have.

Broadly, the paper finds that outside of call centers, there isn’t much low-hanging fruit. It tackles the “fewer adjuncts or smaller classes” conundrum directly, finding that more full-time faculty leads to greater bang for the buck than do smaller classes. (It notes, correctly, that the evidentiary basis for this claim is thin, but at least it’s something.) Given the choice between full-timers teaching large sections and adjuncts teaching small ones, this paper suggests the former. Strikingly, it notes that the bang-for-the-buck of most student services and student service programs is terrible. The TRIO programs look particularly bad, with Upward Bound standing out as a conspicuous boondoggle.

I’m not sure I buy every argument in the paper, but it’s a great start.

It leaves out a critical factor in administrative decision-making, though, which is the sources of money for the various programs. If every dollar came from the same pot, then the bang-for-the-buck measure would be significant. But some dollars come from pots of their own. If a program is entirely grant-funded, and those grant dollars can only be used for that program, then the fact that those dollars could have been more productively used elsewhere is of only theoretical interest. (Put differently, if the choice is between program A and program B, that’s one thing. If it’s between program A and nothing at all, that’s something else.) As long as, say, Federal dollars will pay the entire cost for a given program, then my only concern is whether the program makes any positive difference at all. The question of relative payoff may make sense at, say, the Congressional level, but not here.

There’s also the question of the financial relevance of cost-per-degree. (I’ll leave aside the educational relevance, since that’s too easy. Yes, a degree should actually signify something. That’s why we need robust outcomes assessment. Noted.) My college, like most, doesn’t get funded by the degree. It gets funded by a set amount given by the legislature, plus student tuition. (It also subsidizes the credit-bearing side by profits from workforce development contracts, but that’s neither here nor there.) Tuition is by the credit, and the state appropriation is by the whim of the legislature and governor. In other words, while I can acknowledge the inherent goodness of student success, improved success may not pay for itself locally. In fact, it almost certainly won’t, since tuition covers less than the cost of educating a student, and graduation rates don’t affect our appropriation in any intelligible way.

That’s why I can understand the argument about full-time faculty, but not be able to do much with it. The dollars simply aren’t there. (Of course, we could try to divert money from, say, counselors to faculty, so the analysis isn’t entirely useless.) This is in contrast to the for-profits, where tuition more than pays for its attendant costs. That’s why the for-profits are as focused on student success as they are, and why they pioneered call centers. They capture their own gains. We don’t. Cost effectiveness is great, but if the cost accrues to the college and the benefits don’t, we can expect underinvestment. Doing otherwise would be irrational.

Notably, the study notes that the underlying “cost disease” of higher education as currently defined -- broadly, denominating currency in units of time -- more than swamps any savings to be had by adopting even the most rigorous use of comparative measures. (It also assumes, I think falsely, that call centers and similar “intrusive advisement” models have no negative effect on academic quality. Based on what I saw at Proprietary U, I’d suggest that students who discover that the college needs them more than they need the college will adopt attitudes of entitlement that will make academic rigor more of an uphill battle.) In other words, it addresses more intelligent short-term decisionmaking, rather than fundamental structural change. That’s useful as far as it goes, but it only goes so far.

Still, caveats noted, I have to give thumbs-up overall. Subjecting the claims of various campus constituencies to evidence-based analysis strikes me as worth trying. Diverting money from boondoggles to productive uses may not make all the difference, but it would certainly help. As someone who actually has to make certain budgetary decisions, I say thanks.