Unfortunately, some of us are old enough to have passed through various incarnations of the accountability movement in higher education. Periodically university people or their critics rediscover the notion of accountability, as if the notion of being accountable to students, parents, legislators, donors, federal agencies, and other institutional constituencies were something new and unrecognized by our colleagues. We appear to have entered another cycle, signaled by the publication last month of a call to action by the State Higher Education Executive Officers (SHEEO) association, with support from the Ford Foundation, called "Accountability for Better Results."
The SHEEO report has the virtue of recognizing many of the reasons why state-level accountability systems fail, and focuses its attention primarily on the issue of access and graduation rates. While this is a currently popular and important topic, the SHEEO report illustrates why the notion of "accountability" by itself has little meaning. Universities and colleges have many constituencies, consumers, funding groups, interested parties, and friends. Every group expects the university to do things in ways that satisfy their goals and objectives, and seek "accountability" from the institution to ensure that their priorities drive the university’s performance. While each of these widely differentiated accountability goals may be appropriate for each group, the sum of these goals do not approach anything like "institutional accountability."
Accountability has special meaning in public universities where it usually signifies a response to the concerns of state legislators and other public constituencies that a campus is actually producing what the state wants with the money the state provides. This is the most common form of accountability, and often leads to accountability systems or projects that attempt to put all institutions of higher education into a common framework to ensure the wise expenditure of state money on the delivery of higher education products to the people.
In this form, accountability is usually a great time sink with no particular value, although it has the virtue of keeping everyone occupied generating volumes of data of dubious value in complex ways that will exhaust the participants before having any useful impact. The SHEEO report is particularly clear on this point.
This form of accountability has almost no practical utility because state agencies cannot accurately distinguish one institution of higher education from the other for the purposes of providing differential funding. If the state accountability system does not provide differential funding for differential performance, then the exercise is more in the nature of an intense conversation about what good things the higher education system should be doing rather than a process for creating a system that could actually hold institutions accountable for their performance.
Public agencies rarely hold institutions accountable because to do so requires that they punish the poor performers or at least reward the good performers. No institution wants a designation as a poor performer. An institution with problematic performance characteristics as measured by some system will mobilize every political agent at its disposal (local legislators, powerful alumni and friends, student advocates, parents) to modify the accountability criteria to include sufficient indicators on which they can perform well.
In response to this political pressure, and to accommodate the many different kinds, types and characteristics of institutions, the accountability system usually ends up with 20, 30 or more accountability measures. No institution will do well on all of them, and every institution will do well on many of them, so in the end, all institutions will qualify as reasonably effective to very effective, and all will remain funded more or less as before.
The lifecycle of this process is quite long and provides considerable opportunity for impassioned rhetoric about how well individual institutions serve their students and communities, how effective the research programs are in enhancing economic development, how valuable the public service activities enhance the state, and so on. At the end, when most participants have exhausted their energy and rhetoric, and when the accountability system has achieved stasis, everyone will declare a victory and the accountability impulse will go dormant for several years until rediscovered again.
Often, state accountability systems offer systematic data reporting schemes with goals and targets defined in terms of improvement, but without incentives or sanctions. These systems assume that the value of measuring alone will motivate institutions to improve to avoid being marked as ineffective. This kind of system has value in identifying the goals and objectives of the state for its institutions, but often relegates the notion of accountability to the reporting of data rather than the allocation of money, where it could make a significant difference.
If an institution, state, or other entity wants to insist on improved performance from universities, they must specify the performance they seek and then adjust state appropriations to reward those who meet or exceed the established standard. Reductions in state budgets for institutions that fail to perform are rare for obvious political reasons, but the least effective system is one that allocates funds to poorly performing institutions with the expectation that the reward for poor performance will motivate improvement. One key to effective performance improvement, reinforced in the SHEEO report, is strictly limiting the number of key indicators for measuring improvement. If the number of indicators exceeds 10, the exercise is likely to find all institutions performing well on some indicator and therefore all deserving of continued support.
Often the skepticism that surrounds state accountability systems stems from a mismatch between the goals of the state (with an investment of perhaps 30 percent or less of the institutional budget) and those of the institutions. Campuses may seek nationally competitive performance in research, teaching, outreach, and other activities. States may seek improvement in access and student graduation rates as the primary determinants of accountability. Institutions may see the state’s efforts as detracting from the institution’s drive toward national reputation and success. Such mismatches in goals and objectives often weaken the effectiveness of state accountability programs.
Universities are very complex and serve many constituencies with many different expectations about the institutions’ activities. Improvement comes from focusing carefully on particular aspects of an institution’s performance, identifying reliable and preferably nationally referenced indicators, and then investing in success. While the selection of improvement goals and the development of good measures are essential, the most important element in all improvement programs is the ability to move money to reward success.
If an accountability system only measures improvement and celebrates success, it will produce a warm glow of short duration. Performance improvement is hard work and takes time, while campus budgets change every year. Effective measurement is often time consuming and sometimes difficult, and campus units will not participate effectively unless there is a reward. The reward that all higher education institutions and their constituent units understand is money. This is not necessarily money reflected in salary increases, although that is surely effective in some contexts.
Primarily what motivates university improvement, however, is the opportunity to enhance the capacity of a campus. If a campus teaches more students, and as a result earns the opportunity to recruit additional faculty members, this financial reward is of major significance and will motivate continued improvement. At the same time, the campus that seeks improvement cannot reward failure. If enrollment declines, the campus should not receive compensatory funding in hopes of future improvement. Instead, a poorly performing campus should work harder to get better so it too can earn additional support.
In public institutions, the small proportion of state funding within the total budget limits the ability of state systems to influence campus behavior by reallocating funding. In particular, in many states, most of the public money pays for salaries, and reallocating funds proves difficult. Nonetheless, most public systems and legislatures can identify some funds to allocate as a reward for improved performance. Even relatively small budget increases represent a significant reward for campus achievements.
Accountability, as the SHEEO report highlights, is a word with no meaning until we define the measures and the purpose. If we mean accountability to satisfy public expectations for multiple institutions on many variables, we can expect that the exercise will be time consuming and of little practical impact. If we mean accountability to improve the institution’s performance in specific ways, then we know we need to develop a few key measures and move at least some money to reward improvement.
John V. Lombardi
John V. Lombardi, chancellor and professor of history at the University of Massachusetts Amherst, writes Reality Check every two weeks. Scott McLemee's column, Intellectual Affairs, will return Thursday.
Over the last generation, most colleges and universities have experienced considerable grade inflation. Much lamented by traditionalists and explained away or minimized by more permissive faculty, the phenomenon presents itself both as an increase in students’ grade point averages at graduation as well as an increase in high grades and a decrease in low grades recorded for individual courses. More prevalent in humanities and social science than in science and math courses and in elite private institutions than in public institutions, discussion about grade inflation generates a great deal of heat, if not always as much light.
While the debate on the moral virtues of any particular form of grade distribution fascinates as cultural artifact, the variability of grading standards has a more practical consequence. As grades increasingly reflect an idiosyncratic and locally defined performance levels, their value for outside consumers of university products declines. Who knows what an "A" in American History means? Is the A student one of the top 10 percent in the class or one of the top 50 percent?
Fuzziness in grading reflects a general fuzziness in defining clearly what we teach our students and what we expect of them. When asked to defend our grading practices by external observers -- parents, employers, graduate schools, or professional schools -- our answers tend toward a vague if earnest exposition on the complexity of learning, the motivational differences in evaluation techniques, and the pedagogical value of learning over grading. All of this may well be true in some abstract sense, but our consumers find our explanations unpersuasive and on occasion misleading.
They turn, then, to various forms of standardized testing. When the grades of an undergraduate have an unpredictable relevance to a standard measure performance, and when high quality institutions that should set the performance standard routinely give large proportions of their students “A” grades, others must look elsewhere for some reliable reference. A 3.95 GPA should reflect the same level of preparation for students from different institutions.
Because they do not, we turn to the GMAT, LSAT, GRE, or MCAT, to take four famous examples. These tests normalize the results from the standards-free zone of American higher education. The students who aspire to law or medical school all have good grades, especially in history or organic chemistry. In some cases, a student’s college grades may prove little more than his or her ability to fulfill requirements and mean considerably less than the results of a standardized test that attempts to identify precisely what the student knows that is relevant to the next level of academic activity.
Although many of us worry that these tests may be biased against various subpopulations, emphasize the wrong kind of knowledge, and encourage students to waste time and money on test prep courses, they have one virtue our grading system does not provide: The tests offer a standardized measure of a specific and clearly defined subset of knowledge deemed useful by those who require them for admission to graduate or professional study.
Measuring State Investment
If the confusion over the value of grades and test scores were not enough, we discover that at least for public institutions, our state accountability systems focus heavily on an attempt to determine whether student performance reflects a reasonable value for taxpayer investment in colleges and universities. This accountability process engages a wide range of measures -- time to degree, graduation rate, student satisfaction, employment, graduate and professional admission, and other indicators of undergraduate performance -- but even with the serious defects in most of these systems, they respond to the same problems as do standardized tests.
Our friends and supporters have little confidence in the self-generated mechanisms we use to specify the achievement of our students. If the legislature believed that students graduating with a 3.0 GPA were all good performers measured against a rigorous national standard applied to reasonably comparable curricula, they would not worry much about accountability. They would just observe whether our students learned enough to earn a nationally normed 3.0 GPA.
Of course, we have no such mechanism to validate the performance of our students. We do not know whether our graduates leave better or worse prepared than the students from other institutions. We too, in recognition of the abdication of our own academic authority as undergraduate institutions, rely on the GRE, MCAT, LSAT, and GMAT to tell us whether the students who apply (including our own graduates) can meet the challenges of advanced study at our own universities.
Partly this follows from another peculiarity of the competitive nature of the American higher education industry. Those institutions we deem most selective enroll students with high SATs on average (recognizing that a high school record is valuable only when validated in some fashion by a standardized test). Moreover, because selective institutions admit smart students who have the ability to perform well, and because these institutions have gone to such trouble to recruit them, elite colleges often feel compelled to fulfill the prophecy of the students’ potential by ensuring that most graduate with GPA’s in the A range. After all, they may say, average does not apply to our students because they are all, by definition, above average.
When reliable standards of performance weaken in any significant and highly competitive industry, consumers seek alternative external means of validating the quality of the services provided. The reluctance of colleges and universities, especially the best among us, to define what they expect from their students in any rigorous and comparable way, brings accreditation agencies, athletic organizations, standardized test providers, and state accountability commissions into the conversation, measuring the value of the institution’s results against various nationally consistent expectations of performance.
We academics dislike these intrusions into our academic space because they coerce us to teach to the tests or the accountability systems, but the real enemy is our own unwillingness to adopt rigorous national standards of our own.
American colleges and universities, especially those that define themselves as public institutions because they are owned by states, carry on a continuous conversation with their faculty, students, trustees, legislators, alumni and friends about the distribution of benefits and costs between private and public entities. This conversation of many decades has gained considerable visibility lately in the form of a question: Are America’s public universities becoming private? Although this question is surely worth the extended and often highly perceptive analysis it receives, it sometimes helps to reconfigure the debate slightly to gain another perspective.
It’s not that anyone misses the central point -- the public, tax supported percentage of public university budgets has been in decline for over a decade, even though the public investment in public higher education in total dollars continues to rise as more and more students enter postsecondary education. Rather, we often let our words define our view of the world when our words may not mean exactly what we take them to mean.
When we say public universities, we immediately bring a prototypical institution to mind, usually a substantial state flagship university, often from a Midwestern frame of reference, perhaps modeled after Iowa or Indiana or Wisconsin. When we say private university we also have a prototype in mind, perhaps Stanford, Yale or Duke. From these prototypes we develop a conversation about the convergence of public and private that leads us to worry about the loss of public purpose and investment in American higher education.
In the real world, most of public higher education takes place in state and community colleges that remain often 80 to 90 percent funded by public sources. For these institutions, the issue of public versus private is mostly irrelevant, and while they celebrate every small gift and modest grant, their primary focus is on their states and localities in the endless effort to sustain their operations. They are not at risk of becoming private.
Similarly, in the real world, the notion of private universities being somehow separate and independent from the obligations of public institutions by virtue of their funding sources is also not entirely accurate. Private universities, even those with exceptional endowments, exist to large extent on the public’s account. Their endowments succeed by virtue of public tax exemptions. The gifts that build the endowment enjoy a public tax exemption. The property and campuses of these private universities enjoy a public tax exemption. The federal government provides extensive tax supported need based financial aid to private institutions, revenue that subsidizes those institutions’ tuition and fess.
Private research universities, like their public counterparts, receive federal grants and contracts whose overhead pays some portion of the research costs, a direct taxpayer subsidy. Private universities in many states receive a per-student subsidy for every in-state student they enroll, again a public subsidy. And on occasion, private universities succeed in persuading their states to invest in economic development activities that support the academic objectives of the private institution (either by subsidizing research or helping defray the costs of facilities).
America’s private institutions are a public trust. While they can evade many of the considerable bureaucratic and regulatory costs and obligations that public universities endure, they are nonetheless, publicly subsidized institutions with private governance.
This is not a bad thing. It is just how we do business in America.
However, higher education itself (public or private as defined by institutional governance) is both a public good and a private good for most of its participants. Students in particular may attend college for wisdom and knowledge, but primarily they attend college to acquire the skills and credentials needed for the good life. Publication after publication calculates and compares the differential lifetime earnings of college graduates compared to high school graduates, demonstrating over and over again the exceptionally high personal, private value that a college education confers. The private benefit justifies tuition and fees, the lost income for the years of college attendance, and the loan indebtedness incurred by students and their families. The data tell us that these private benefits more than sufficiently compensate for the costs parents and students assume, a conclusion the behavior of students and parents verifies.
What’s the argument about then? If this is such a good deal, why do we have a controversy about the withdrawal of public support from public institutions? The controversy is less about the withdrawal of support than it is about the amount of subsidy individuals should receive as they acquire the private benefit of a college education. The consequence of a decline in the taxpayer subsidy of public higher education is an increase in the net cost of higher education to students. This increase in the net cost has many consequences, of course. As an example, for students whose families are at the margin of the American economic dream, any increase in the net cost may well put some forms of higher education, but usually not all forms, out of reach.
The changing emphasis away from the general benefit colleges and universities bring society to the particular benefit that they bring individuals encourages a tendency toward complex pricing. For selective institutions, as everyone knows, the sticker price of higher education (whether at a “public” or “private” institution) reflects what we could call a reference price. This is not the actual price charged every student; instead, a reference price marks the highest price a student should have to pay to acquire the private benefit of attending the institution. To arrive at the real price, the institution and the student engage in a private negotiation to set the net actual price based on an evaluation of what the institution can give the individual student and what the individual student can give the institution. This is not a public transaction that applies to all students; it is a private transaction that negotiates a private price between suppliers (the school) and individual consumers (the student).
Although this transactional model is well known to students and parents, it reflects a larger tendency in the American political and social environment to disaggregate a general public good (such as a university enterprise) into a collection of private goods (such as specific college degrees, different majors, or special programs) and then negotiate separate agreements and pricing mechanisms for the production and delivery of these private goods.
Many public and private universities find it easier to persuade legislators to buy particular fragments of the institution’s purposes than it is to acquire general funding for the overall purpose of the institution. We can get an earmark for an honors program, for a remedial program for students from disadvantaged backgrounds, for enhancement of science and math, for the improvement of writing skills, or for a research building tied to a specific economic development objective long before we can get an increase in the general fund to support general education and research for all students and faculty.
This particularistic approach to higher education affects public and private institutions in other areas of funding as well. When either type of institution asks a donor for institutional support, more and more donors want very specific agreements about the exact use of their funds, even if the funds are placed in an endowment that will last forever. They do not give money for the improvement of education; they give funds for art history, for the development of specific scientific sub-disciplines, or for the recruitment of basketball players. These transactions, like the student transactions, are individual, private, and specific.
We can speculate on the many reasons why our society has drifted into seeing higher education as a retail consumer product (whether owned by the public or by a private nonprofit corporation). We can worry about the lack of faith in the institutions’ integrity and consistency of purpose that encourages specific and detailed transactions rather than satisfaction with general commitments. We can feel outrage at the retreat from supporting higher education because it is good for the nation into a safe haven that sees higher education as a privately acquired ticket to prosperity. Yet every institution, public or private, finds itself accelerating these trends by its policies and practices.
I’m often reminded of an effort some years ago by a distinguished association of public and private research institutions to band together and refuse to participate in such retail negotiations, associated in this instance with federal earmarks. The academic leadership at that time employed remarkable eloquence in the defense of common approaches to peer reviewed grant making as being the best possible means of achieving the public good of merit based award of scientific and other educational support. After the meeting, the institutions appeared to increase their practice of securing as many earmarks in the federal budget as possible, employed high powered lobbyists to improve their chances, and kept score on how much money their local legislators helped them bring home.
All of us in American higher education, especially at the high end of America’s 170 or so research institutions, are in the public and private sectors. We all seek public funds and private funds, we all deal with our students on the basis of selling a publicly subsidized product at individually negotiated prices, we all show great creativity in disaggregating our products and services into the smallest retail units needed for sale to our many private purchasers. We might prefer a different system, but this one, for all its faults, is the one we’ve helped invent and continue to refine.