Do Colleges Measure What They Value?

Study examines what colleges say they want students to learn and how they are measuring that learning.

May 1, 2019

As Americans express growing doubts about the value of a postsecondary degree, colleges and universities have been under increasing pressure to show that students emerge with the knowledge and/or skills the institutions say they're trying to develop. Not everyone applauds the push to measure student learning, but the pressure to be more intentional about the outcomes a college or program aims to develop isn't likely to abate soon.

A new report, "Degree of Difference: What Do Learning Outcomes Say About Higher Education?" digs into data about "learning outcome statements" at dozens of colleges and universities to see what institutions say they want their students to be learning and how they measure whether that learning occurred.

Its conclusions: many colleges don't align what they're trying to do at the program and department level with an overall institutional approach.

The study, from Campus Labs, which makes data and assessment software for higher education, taps into its database of the learning outcome statements that colleges produce (typically for accreditation purposes) to show the desired goals of a course, an academic program or a school or campus. The data come from 73 colleges and universities, a mix of two-year and four-year.

The study sorted about 8,500 outcomes found in those statements into broad "themes" to try to gauge which outcomes were valued most at the institutional and program levels. As seen in the chart below, "intellectual skills" -- incorporating problem solving, critical thinking and reasoning -- was the most-cited outcome in both institutionwide and department or program outcome statements.

The authors said they were surprised to see greater emphasis on personal development in student outcomes at the program level than at the institutional level.

Drilling down, the report shows differences at the two- and four-year level, as seen below. The community colleges' focus on technology -- which includes things such as technical literacy and fluency, computer science, and emerging technologies -- is consistent with the mission of those institutions, the authors write.

The report also explored how institutions in the sample went about measuring whether students were achieving the outcomes. The most commonly used method of gauging achievement of a particular outcome was a survey questionnaire, used in 16 percent of cases, especially for outcomes such as intellectual skills, personal development and technology.

This was followed, in descending order, by essays, exams, final exams and individual projects. Essays were the most often used assessment for things such as communication, creative arts and humanities.

Perhaps the biggest concern cited by the researchers was the emphasis (or lack thereof) that institutions seemed to put on quantitative reasoning, which appeared far down the list of outcomes that colleges and programs sought to measure.

"There is no other way to say it -- campuses need to up their game in quantitative reasoning. Employers have consistently listed analytical and quantitative skills as a desirable attribute, yet according these data, the quantitative learning theme shows up in only 2 percent of outcome statements and is most often measured by exams," the authors write.

Share Article

Doug Lederman

Doug Lederman is editor and co-founder of Inside Higher Ed. He helps lead the news organization's editorial operations, overseeing news content, opinion pieces, career advice, blogs and other features. Doug speaks widely about higher education, including on C-Span and National Public Radio and at meetings and on campuses around the country, and his work has appeared in The New York Times and USA Today, among other publications. Doug was managing editor of The Chronicle of Higher Education from 1999 to 2003. Before that, Doug had worked at The Chronicle since 1986 in a variety of roles, first as an athletics reporter and editor. He has won three National Awards for Education Reporting from the Education Writers Association, including one in 2009 for a series of Inside Higher Ed articles he co-wrote on college rankings. He began his career as a news clerk at The New York Times. He grew up in Shaker Heights, Ohio, and graduated in 1984 from Princeton University. Doug lives with his wife, Kate Scharff, in Bethesda, Md.

Back to Top