July 31, 2007

The Quest for ‘Meaningful’ Comparisons

When college leaders criticize rankings or fend off questions about accountability, they frequently point to the National Survey of Student Engagement. NSSE (pronounced like the Loch Ness monster’s nickname) surveys the views of first-year and fourth-year students on a variety of measures of engagement and colleges say that they use the detailed reports they receive to improve their offerings. There’s one major problem from an accountability perspective: NSSE lets colleges decide whether to make public any or all of their data, so while national averages are released every year, it can be difficult or impossible for a prospective student to know how a college is doing.

Looking for a job?

See all new postings

Browse all job listings:

Faculty: 3,481
Administrative: 1,574
Executive: 156
FEATURED EMPLOYERS

NSSE has just asked all of the colleges that participated at least once in the last three years (a total of about 1,000 four-year institutions) for permission to release five benchmark scores from the engagement survey’s report for their institutions to USA Today, which may place the data online. Both NSSE and USA Today have stressed that they are not trying to create a new ranking system, and that institutions would not be ranked. Rather, the effort is an attempt to provide more “meaningful indicators” about colleges that students and parents could use.

The five benchmarks are: level of academic challenge, active and collaborative learning, student-faculty interaction, enriching educational experiences, and supportive campus environments.

Initial responses to the NSSE request suggest that many colleges would make their data public. George D. Kuh, director of NSSE and of the Center for Postsecondary Research at Indiana University at Bloomington, said that he has heard from about a quarter of the participating colleges. Of those, he said that about half have indicated that they would make their data public, about one quarter have said “maybe,” and the remainder have said that they aren’t interested in doing so at this time.

An FAQ prepared by NSSE for participants is emphatic on the idea that this project is not about rankings. “The project is intended to respond to calls for greater institutional transparency and to underscore the idea that educational quality is more complex than typically available elsewhere, such as in rankings,” the statement said. “The focus will be on informing people with an interest in collegiate quality about the indicators of educational effectiveness represented by NSSE benchmarks and items as well as distinctive patterns of engaging educational activities offered by different types of institutions around the country.”

In an interview, Kuh stressed that NSSE benchmark scores wouldn’t yield a good rankings system in any case. “Very few schools score high across all dimensions,” he said. The value in making the data public is that people can see particular strengths and also learn which institutions might have models worth studying. “There are a lot of good things going on in higher education that people don’t know about, some of them right around the corner.”

NSSE has never opposed the release of its data by colleges, and some do so, but institutions were promised from the beginning that individual institutions’ statistics would not be released without their permission. Kuh said that the board of the organization “wants to help institutions move along” toward greater disclosure. Kuh also said he saw the move toward releasing data as something that would add to efforts being pursued by several college associations to make more information about colleges publicly available.

The Community College Survey of Student Engagement, a similar project for two-year colleges which is based at the University of Texas at Austin, makes all data public. But almost all community colleges are public while many NSSE participants are private, and rankings have been a much more sensitive issue for four-year institutions.

Alexandra Nicholson, communications manager for USA Today, said that the newspaper is still in the “information gathering” stage of the project and has no firm plans on how the data would be used. She said, however, that the idea “was not to create rankings.”

Kuh said that some of the institutions that have said “maybe” — and presumably some of those that haven’t responded yet — are waiting for this year’s scores, which are about to be sent to individual colleges. But the FAQ makes a case to institutions that they should participate even if they aren’t thrilled with their scores this year.

“Schools that do participate in this initiative will be able to declare and demonstrate their commitment to improving and being accountable for undergraduate education,” the FAQ said, adding that the NSSE board believes that “the time has come for participating institutions to stand together in promoting responsible ways to make available information about important, relevant features of institutional and student performance, and to continue to provide leadership for improving the quality of undergraduate education.”

— Scott Jaschik

Want it on paper? Print this page.
Know someone who’d be interested? Forward this story.
Want to stay informed? Sign up for free daily news e-mail.

Advertisement

Comments