Research News

Paper from UW’s Hillman explains that performance-based funding is falling short

January 14, 2015
by American Educational Research Association

Performance-based funding is falling short of its intended goals of raising student retention and degree completion rates at community colleges, according to a new research paper co-authored by UW-Madison's Nicholas Hillman.

The paper, titled “Evaluating the Impacts of ‘New’ Performance Funding in Higher Education,” was published Wednesday in Educational Evaluation and Policy Analysis, a peer-reviewed journal of the American Educational Research Association (AERA). The results of the study, which focused on the state of Washington’s Student Achievement Initiative (SAI) — widely recognized as a model for performance accounting systems in the United States — are in line with recent results in Tennessee and Pennsylvania indicating that performance funding has not improved retention and graduation rates.

Hillman is an assistant professor in the Department of Educational Leadership and Policy Analysis, and an affiliate of the Wisconsin Center for the Advancement of Postsecondary Education (WISCAPE)

Using data for 2002 through 2012 from the U.S. Department of Education’s Integrated Postsecondary Education Data System, authors Hillman, David A. Tandberg of Florida State University, and Alisa Hicklin Fryar of the University of Oklahoma, found that retention and degree completion at Washington’s community and technical colleges were not distinguishable from the performance of colleges in states without similar accountability policies.

Hillman AERA video
In this video, Nicholas Hillman discusses the key
findings from the research paper.
“Tying state financing to college ‘performance’ is expected to create an incentive for colleges to produce more degree recipients and retain students at higher rates,” Hillman says in an AERA news release that explains the paper’s key findings. “However, evidence from the Washington experiment suggests these results do not occur in systematic ways.”

Introduced in 2007, Washington’s initiative is one of the nation’s longest-operating performance funding programs currently in place, and it has been widely recognized as one of the most carefully designed performance accountability systems in the country.

“Considering the popularity of Washington’s performance funding model, we are surprised the impacts on associate’s degree productivity are so modest,” write the authors.

While Washington did not experience a systematic increase in retention and associate’s degree production following the implementation of SAI, it did see noticeable growth in students earning short-term certificates which, overall, hold little value in the labor market.

The authors noted that original SAI guidelines gave community colleges incentive to expand short-term certificate programs, which are designed to be completed in less than one year, even though it was not one of the primary goals of the state’s policy.

“Short-term certificates are an easier approach than two-year completion or graduation. They’re the path of least resistance for schools.” Tandberg says in the AERA news release. “However, short-term certificates often don’t provide any benefits over a high school diploma.”

In December 2012, Washington’s policy was revised and now awards achievement points only for short-term certificates that lead to middle and higher wage jobs or to a more advanced credential.

The authors noted that since the mid-2000s, performance funding programs, which fund colleges based on institutional “output” measures—including retention and graduation rates—rather than “inputs” such as the number of students enrolled, have gained in popularity, in part due to the efforts of a number of influential national organizations, including the Bill & Melinda Gates Foundation, Complete College America, the Lumina Foundation, the National Conference of State Legislatures, and the National Governors Association.

The authors said that the difficulty of retaining students may be a factor more complicated than SAI allows for, noting that improving retention rates requires significant campus resources to address a broad range of factors, including student engagement levels, academic support services, campus climate, student satisfaction, and financial aid.

“There is no easy solution to improving college performance,” Hillman says in the AERA news release. “Most schools do not have the capacity to make improvements with current resources. That’s especially true of community and technical colleges, which are already known for having to do the most with the least amount of resources.”

To learn more about some of the key findings of this new research, check out this video interview with Hillman.