The Century Foundation on Wednesday published a report that is critical of state policies that link funding of public colleges with measures of their performance, such as graduation rates and degree production numbers. Roughly 35 states are either developing or using some form of performance-based funding for higher education.
The new report's author, Nicholas Hillman, an assistant professor of education at the University of Wisconsin at Madison who has studied such state-based formulas, argues that performance-based funding is rarely effective.
"While pay for performance is a compelling concept in theory, it has consistently failed to bear fruit in actual implementation, whether in the higher education context or in other public services," Hillman wrote. "Performance-based funding regimes are most likely to work in noncomplex situations where performance is easily measured, tasks are simple and routine, goals are unambiguous, employees have direct control over the production process, and there are not multiple people involved in producing the outcome."
The Institute for Higher Education Policy is today releasing a series of papers that, taken together, are designed to point the way toward a more vibrant set of national data on student outcomes.
The papers, which come from a wide range of policy experts, cover an array of topics, such as the possibility of creating a federal student-level data system, how to link existing federal data systems, strategies for protecting privacy of students and the possible role of the National Student Clearinghouse. The release is in conjunction with an event today in Washington, D.C.
The U.S. Education Department on Monday announced that it had chosen 44 colleges for an experiment in which they will be able to give Pell Grants to high school students participating in dual enrollment programs. The announcement carries out the department's plan (another in a string of efforts to use its "experimental sites" authority) to allow as many as 10,000 high school students to use federal postsecondary student aid funds to take college-level courses, which is generally prohibited by federal law.
The institutions chosen to participate, about 80 percent of which are community colleges, have agreed to use promising practices for ensuring the students' success, such as creating clear curricular pathways, building linkages to careers and ensuring strong advising.
A new report from the Institute for Higher Education Policy identifies the key metrics that would help federal and state data systems provide information on colleges' performance, efficiency and equity.
The report, developed in partnership with the Bill and Melinda Gates Foundation, details that the information provided today leaves out answers to college access, progression, completion, cost and outcomes. Using the three metrics identified in the report and integrating them into federal and state systems will make the information available to all students from all types of institutions.
"This report draws on the knowledge and experience of higher education leaders and experts to lay out in detail the metrics we should be collecting and explains why those data will make a difference, for all students, but particularly for those who traditionally have been underserved by higher education," said Michelle Cooper, IHEP's president, in a news release. "The field needs a core set of comprehensive and comparable metrics and should incorporate those metrics into federal and state data systems."
General Assembly, the largest of the skills boot camp providers, today released a public framework for measuring student outcomes. Boot camps are not accredited. And while many claim job-placement rates of more than 90 percent, those numbers typically are not verified by outside groups. But Skills Fund, a student lender for boot camps, and other players are seeking to play that role.
To design its standards for reporting and measuring student success, General Assembly worked with two major accounting firms to craft an approach public companies use to measure nonfinancial metrics such as social impact and environmental sustainability.
"Our goal is to start a conversation about outcomes predicated on the use of consistent definitions and the application of a rigorous framework and methodology," the company said. "Over time, we hope to develop new measures of return on education that consider income or other criteria that can be used by students and other stakeholders to understand student success in even more specific and granular ways."
A recent Inside Higher Edarticle about the analysis of state performance funding formulas by Seton Hall University researchers Robert Kelchen and Luke Stedrak might unfairly lead readers to believe that such formulas are driving public colleges and universities to intentionally enroll more students from high-income families, displacing much less well-off students. It would be cause for concern if institutions were intentionally responding to performance-based funding policies by shifting their admissions policies in ways that make it harder for students who are eligible to receive Pell Grants to go to college.
Kelchen and Stedrak’s study raises this possibility, but even they acknowledge the data fall woefully short of supporting such a conclusion. These actions would, in fact, be contrary to the policy intent of more recent and thoughtfully designed outcomes-based funding models pursued in states such as Ohio and Tennessee. These formulas were adopted to signal to colleges and universities that increases in attainment that lead to a better-educated society necessarily come from doing a much better job of serving and graduating all students, especially students of color and students from low-income families.
Unfortunately, Kelchen’s study has significant limitations, as has been the case with previous studies of performance-based funding. Most notably, as acknowledged by Kelchen and Stedrak, these studies lump together a wide variety of approaches to performance-based funding, some adopted decades ago, which address a number of challenges not limited to the country’s dire need to increase educational attainment. Such a one-size-fits-all approach fails to give adequate attention to the fact that how funding policies are designed and implemented actually matters.
For example, the researchers’ assertion that institutions could possibly be changing admissions policies to enroll better-prepared, higher-income students does not account for differential effects among states that provide additional financial incentives in their formulas to ensure low-income and minority students’ needs are addressed vs. those states that do nothing in this area. All states are simply lumped together for purposes of the analysis.
In addition, the claim that a decrease in Pell dollars per full-time-equivalent student could possibly be caused by performance-based funding fails to account for changes over time in federal policy related to Pell Grants, different state (and institutional) tuition policies, other state policies adopted or enacted over time, changes in the economy and national and state economic well-being, and changes in student behavior and preferences. For example, Indiana public research and comprehensive universities have become more selective over time because of a policy change requiring four-year institutions to stop offering remedial and developmental education and associate degrees, instead sending these students to community colleges.
If any of these factors have affected states with newer, well-designed outcomes-based funding systems and other states with rudimentary performance-based funding or no such systems at all, as I believe they have, then there is strong potential for a research bias introduced by failing to account for key variables. For example, in states that are offering incentives for students to enroll in community colleges, such as Tennessee, the average value of Pell Grants at public bachelor’s-granting institutions would drop if more low-income, Pell-eligible students were to choose to go to lower-cost, or free, community colleges.
I agree with Kelchen and Stedrak that more evaluation and discussion are needed on all forms of higher education finance formulas to better understand their effects on institutional behavior and student outcomes. Clearly, there are states that had, and in some cases continue to have, funding models designed in a way that could create perverse incentives for institutions to raise admissions standards or to respond in other ways that run contrary to raising attainment for all students, and for students of color in particular. As the Seton Hall researchers point out, priority should be given to understanding the differential effects of various elements that go into the design and implementation of state funding models.
The HCM Strategists’ report referenced in the study was an attempt by us to inform state funding model design and implementation efforts. There needs to be a better understanding of which design elements matter for which students in which contexts -- as well as the implications of these evidence-based findings for policy design and what finance policy approaches result in the best institutional responses for students. There is clear evidence that performance funding can and does prompt institutions to improve student supports and incentives in ways that benefit students.
Analysis under way by Research for Action, an independent, Philadelphia-based research shop, will attempt to account for several of the existing methodological limitations correctly noted by Kelchen and Stedrak. This quantitative and qualitative analysis focuses on the three most robust and longest-tenured outcomes-based funding systems, in Indiana, Ohio and Tennessee.
Factors examined by Research for Action will include the type of outcomes-based funding being implemented, specifics of each state’s formula as applied to both the two- and four-year sectors, the timing of full implementation, changes in state policies over time, differences in the percentages of funding allocated based on outcomes such as program and degree completion, and differences in overall state allocations to public higher education. And, for the first time, Research for Action will move beyond the limitations of analyses based primarily on federal IPEDS data by incorporating state longitudinal data, which give a more complete picture.
As states continue to implement various approaches to funding higher education, it is essential to understand the effects on institutional behavior and student outcomes. Doing so will require more careful analyses than those seen to date and a more detailed understanding of policy design and implementation factors that are likely to affect institutional responses. Broad-brush analyses such as Kelchen and Stedrak’s can help to inform the questions that need to be asked but should not be used to draw any meaningful conclusions about the most effective ways to ensure colleges and universities develop and maintain a laser focus on graduating more students with meaningful credentials that offer real hope for the future.
Martha Snyder is a director at HCM Strategists, a public policy advocacy and consulting firm.
The Association to Advance Collegiate Schools of Business, an accreditor, last week released a new "vision" for management education. In a report the group identified five roles business schools are well positioned to fill, including being catalysts for innovation, co-creators of knowledge, hubs of lifelong learning, leaders on leadership and enablers of global prosperity.
"Business education has changed dramatically in the past decade, and schools are facing increasing pressure to drive positive economic and social impact," said Santiago Iniguez de Onzono, chair of AACSB's Committee on Issues in Management Education, chair-elect of the AACSB Board and dean of IE Business School, in a written statement. "Currently many business schools across the globe are already rising to the challenge. By identifying a clearer path forward, the collective vision helps to accelerate the transformation already underway, while becoming a key resource for business schools as they innovate and contribute to society in a more meaningful way."
An ongoing study conducted by Research for Action, a Philadelphia-based nonprofit research organization, looked at the effect of performance-based funding policies in higher education across three states: Indiana, Ohio and Tennessee. The group released early results from the work over the weekend at the annual meeting of the American Education Research Association.
The project takes into account key differences in the type of policies as well as variations in state funding that were tied to them. Initial findings showed consistent positive effects on the numbers of bachelor's degrees awarded under the policies. But the study did not find evidence of a positive effect on graduation rates.