Chris Birdsall is an Assistant Professor of Public Policy and Administration in the School of Public Service at Boise State University. He completed his PhD in 2016 and his MPP in 2012 at American University in Washington, D.C. Prior to his graduate studies, Chris worked as a Legislative Aide in the Alaska Legislature. His research focuses on public management, performance management, and higher education. He has published in the International Public Management Journal and presented papers at numerous conferences including the annual meetings of the Public Management Research Association, Midwest Political Science Association, American Political Science Association, and Association of Public Policy Analysis and Management.
Idaho set a goal to have 60 percent of 25- to 34-year-old Idahoans hold a college degree or certificate by 2020. Governor C.L. “Butch” Otter’s Higher Education Task Force recently voted unanimously on a dozen recommendations designed to help advance the state toward meeting that goal. Among them is a proposal to distribute some of the state’s higher education funding to institutions based on student outcomes.
Thirty-two states have already implemented policies tying funding for public colleges and universities to student outcomes. A growing body of research about these policies reveals important insights and lessons that can help inform the design and implementation of a more effective outcomes based funding system. First, performance information is complex and not always indicative of actual performance. Second, high-stakes performance accountability systems — tied to money– may create perverse incentives that lead to behaviors that undermine the broader policy goals of preparing students for careers and creating an informed citizenry. Finally, policymakers seeking to implement an outcomes-based budgeting system for public higher education can minimize or avoid these pitfalls by being sensitive to differences in institutional missions, capacities, and resources.
PERFORMANCE AMBIGUITY
Policymakers often see performance measurement as an objective method for determining how well programs and organizations work. However, performance information is more ambiguous than many realize. Consider: An organization fails to reach a performance benchmark. Is the failure indicative of a managerial or structural problem? Is it due to contextual factors outside the organization’s control? Would the program have done better if it received more resources and funding? These are critical questions, especially if performance measures have financial stakes. But arriving at credible answers is difficult because countless factors can influence performance. Thus, performance information may contribute little to settling budgeting debates. Critics cite poor performance as a reason to reduce resources dedicated to a program, while supporters argue poor performance indicates a need for additional resources.
An important feature of outcomes-based funding policies in higher education makes the ambiguity issue especially problematic: allocations are automatic. There is no formalized step for humans to contextualize performance information before it is translated into a budget allocation. If an institution does not meet its performance benchmark, it does not receive its money, regardless of whether the failure to meet the goal was due to lack of resources or mismanagement by university administrators.
The potential problems associated with using performance measurement for budgeting do not end with ambiguity. The efficacy of a performance management system relies on an assumption that organizations are not already dedicating resources toward critical student outcomes. The danger of this assumption is that outcomes-based funding policies asking institutions to make gains that are in reality impossible may lead to perverse behaviors that undermine broader policy goals.
PERVERSE INCENTIVES AND UNINTENDED CONSEQUENCES
The history is clear: when poorly designed performance targets are difficult to reach and tied to strong financial incentives, organizations will cheat, often in ways counterproductive to the policy’s original goals.
The history is clear: when poorly designed performance targets are difficult to reach and tied to strong financial incentives, organizations will cheat, often in ways counterproductive to the policy’s original goals. A recent prominent example is the massive scandal in the Veterans Health Administration in which workers kept patients off official waiting lists so the hospitals could meet performance benchmarks tied to waiting times.
Two potential counterproductive behaviors are particularly relevant to the performance debate in public higher education: weakening academic standards and restricting student access. First, lowering academic standards is a relatively simple way to increase graduation rates without making any tangible improvements that contribute to student success. Rather than focus efforts on activities that increase student achievement, institutions end up making it easier for students to pass classes and graduate. So far, although there is no empirical evidence this is happening, information gathered in a series of interviews of faculty and administrators in states with outcomes-based programs shows there are concerns the policies will result in pressure to lower academic standards.
A second concern is that schools will limit access to education. Restricting enrollment to students more likely to graduate may help institutions reach performance benchmarks, even though this may directly work against broader goals to increase educational attainment. The selectivity adversely targets the very populations that potentially benefit the most from access to education.
Policymakers should be cognizant of these issues as they design and implement performance policies. My research investigates whether outcomes-based funding affected student outcomes in Indiana’s public higher education system and finds a significant decrease in acceptance rates. I also find a small but statistically significant decrease in minority enrollment. These findings are consistent with those of a recent qualitative study in which Indiana higher education administrators expressed concerns about the state policy’s effects on student access, particularly for minorities and students from disadvantaged backgrounds.
AVOID ONE-SIZE-FITS-ALL SOLUTIONS
Public institutions in the United States range from regionally based comprehensive universities with broad access missions designed to serve surrounding communities to internationally renowned, highly selective institutions. These institutions educate dramatically different student populations. How can policymakers design outcomes-based funding policies that reward actual performance and minimize perverse incentives? An important first step is avoiding one-size-fits-all solutions. Neglecting to consider differences among institutions is one of the biggest problems with existing public sector performance management systems. If performance measures and goals do not reflect schools’ individual missions and do not account for differences in financial resources, students, and staff, the institutions may resist the system or act irrationally to help them meet performance benchmarks. Managers may even feel justified in subverting a policy if it protects them from serious negative consequences in a system they feel is unfairly stacked against them. Look no further than the massive cheating scandals that emerged in some of the largest U.S. school districts during the No Child Left Behind era.
The dangers of one-size-fits-all systems are especially acute in American public higher education, where institutional diversity is a signature feature. Public institutions in the United States range from regionally based comprehensive universities with broad access missions designed to serve surrounding communities to internationally renowned, highly selective institutions conducting groundbreaking research and attracting high-achieving students both nationally and from around the world. These institutions educate dramatically different student populations, employ different kinds of faculty, and differ greatly in the students they serve, their financial resources, and staff available to meet performance-oriented change. Broad access institutions, for example, admit higher proportions of first-generation and disadvantaged students, who historically have lower graduation and retention rates. These institutions also have fewer resources and less capacity to direct toward tutoring and other initiatives that may help them improve metrics.
My research provides evidence supporting the notion that differences in mission and capacity affect how institutions respond and perform under outcomes-based funding systems. When I examined the implications of organizational differences for the effects of outcomes-based funding in Indiana, which one university president criticized for being a one-size-fits-all policy, I found that less selective institutions highly dependent on state funding for operating revenue saw near-zero or no effects on graduation rates, while more selective, less state-dependent institutions saw modest positive effects. In other words, only the already high achieving institutions saw gains as a result of the policy. If institutions find they cannot succeed under a outcomes-based funding system, they may turn to other measures to reclaim lost revenue. In Michigan, for example, some public institutions seem to be ignoring the state’s outcomes-based funding policy and instead are turning to tuition increases to secure additional revenue, which surely goes against policymakers’ intentions.
MAKING OUTCOMES-BASED FUNDING WORK
Does all of this mean states should avoid or abandon outcomes-based funding systems for public higher education? Certainly not. While the empirical evidence so far suggests the policies have little or no effects on institution performance, most state policies are in the early implementation stages and it is simply too soon to make general statements about their efficacy. Furthermore, a recent paper by Amanda Rutherford and Tom Rabovsky suggests outcomes-based funding policies may become more effective over time.
Perhaps the essential lesson we can draw so far is factors largely outside the control of the institution, such as their student population and financial resources, play determining roles in university performance. Perhaps the essential lesson we can draw so far is factors largely outside the control of the institution, such as their student population and financial resources, play determining roles in university performance. It is critical, then, to understand what is achievable and to recognize that goals and targets that make sense for one institution may not make sense for another. Broad access institutions, for example, face considerably more challenges when it comes to increasing graduation rates. Policymakers who ignore these differences run the risk of introducing perverse incentives leading to unintended consequences.
There is little debate that America’s public higher education institutions need to improve. Graduation rates remain stubbornly low, attendance is unaffordable for many, and the United States is in danger of falling behind the rest of the developed world when it comes to the percentage of adult population with college degrees. Setting goals and rewarding institutions for achieving them will be an important part of changing this state of affairs. Carefully designing a policy that addresses these issues will help policymakers craft outcomes-based funding systems that increase student success and advance broader educational attainment goals that help states compete in an increasingly knowledge-based global economy.