Meeting of the Minds

In one way or another, leaders in higher education have been working for 20 years on trying to find valid and meaningful ways of measuring how well students learn. Although some institutions have developed their own measures, most college officials agree that there has been much less progress in revealing those results to the public and in finding ways to give students information they might use to compare their chances for success across different institutions, a point made bluntly and quite critically in last year's report of the Secretary of Education's Commission on the Future of Higher Education.

The discussion in recent months about how to alter that landscape has at times pitted the federal government against college officials: Education Department officials have accused some college leaders of dragging their feet and refusing to be held accountable for their performance, and many in higher education assert that the government has tried to impose an overly simplistic, overly standardized approach that fails to account for the rich diversity of colleges' missions and students.

Today, the Education Department plans to announce that it is giving three college associations a $2.4 million grant to help them assess existing, and develop new, tests and other tools to measure student outcomes on a wide range of skills. Officials of the Education Department and the three groups -- the Association of American Colleges and Universities, the American Association of State Colleges and Universities, and the National Association of State Universities and Land-Grant Colleges -- herald the development as a breakthrough.

The embrace of the project by the three groups, their leaders said, shows that colleges are unafraid to assess their performance, but want to ensure that the measures used are appropriate, intelligently crafted, and fully represent the many kinds of skills that students need. And the department's financial backing, said Sara Martinez Tucker, U.S. under secretary of education, shows that department officials meant it when they said they want colleges to "do it themselves," rather than have the government do it to them.

"It is my hope and expectation that this represents a transitional moment in both the dialogue about how to measure student learning and, even more important, in the actual practices we use to do that," said Carol Geary Schneider, president of the AACU, which is taking the lead on the project. "The primary responsibility for achieving educational excellence falls on colleges and universities themselves. And through this initiative, over 1,200 colleges and universities in three different associations will come together to move the assessment agenda forward, in ways that respect the best traditions and the most important purposes of American higher education."

Critics of the push for comparing the learning outcomes of institutions and individuals (notably the National Association of Independent Colleges and Universities) have complained that the Spellings Commission and the Education Department have rushed to adopt standardized measures of general education skills (such as writing and critical thinking) that are not up to the task. They have also argued that other crucial desirable "outcomes" of an undergraduate education, such as civic engagement or ethical reasoning, cannot be measured by existing tools, and that some outcome measures popular on campuses -- such as electronic portfolios -- may have more promise than standardized tools.

The proposal put forward by the three associations -- which the Education Department chose over several competing proposals in a competition, Tucker said in an interview -- deals with all three issues. In full, said Schneider, the project aims to find ways to measure the “broad array of learning outcomes that most colleges and universities consider essential to a good education,” such as those AACU laid out in its recent report, “College Learning for the New Global Century.”

The newly funded FIPSE project contains three main parts. NASULGC will lead the way on a project that will review the effectiveness of three major standardized measures of general education skills that are part of the Voluntary System of Accountability that the land-grant association and AASCU are crafting. Experts from the organizations that sponsor the Collegiate Learning Assessment, the Measure of Academic Progress and Proficiency and the Collegiate Assessment of Academic Proficiency will work together with other testing experts to assess the tests’ relative reliability and validity, so that “you will know it means when you score high or low on one of these tests,” said David Shulenburger, vice president for academic affairs at NASULGC.

“When you’ve got something new, like measuring critical thinking, the academy is skeptical, and it wants lots of testing done before we start making decisions on it,” he said. “We want it done right and we want to measure this by the standards that we use to measure the other kinds of research that we do. This grant will allow us to do that, to fill in that gap.”

AASCU’s part of the project (in which AACU will also participate) will involve an attempt to try to develop tools for measuring student outcomes that aren’t easily measured, such as civic engagement, teamwork, personal and social responsibility, and the like. This effort aims to respond to the idea that many of the traits that a good liberal arts or general education develops are hugely important, but not easily measurable and hence difficult to assess. The goal would be to develop metrics or “rubrics” that colleges could use to measure some of those traits in their students.

The third piece of the project, on which the Association of American Colleges and Universities will take the lead, seeks to tap into the significant work that many colleges have done to try to measure their students’ development using electronic portfolios. Many departments, schools and colleges have developed ways of using collections of student work to show progress not only on general skills like writing but on those tied to their fields of study. AACU’s goal, said Schneider, is to assess the quality of existing e-portfolio assessments on campuses and further develop and share the best ones.

Tucker, the U.S. under secretary, said the decision to finance the project follows on Education Secretary Margaret Spellings’ promise a year ago that she would “explore incentives to states and institutions to collect learning outcomes data.” The Spellings Commission’s report argued strenuously that the country needed better information about colleges’ performance in educating students, and needed it sooner rather than later, Tucker said.

But the department’s efforts to carry out the panel’s recommendations have been misinterpreted as seeking a government, or a “one size fits all,” solution, she said. “The report said that we need colleges to produce better information, and this signals what we’ve said all along: That we’re not doing it to them, we’re doing it with them.”

Tucker said she was pleased at the broad range of institutions – public and private, two-year and four-year -- that will be participating in this project through their associations, and by the relatively tight timeline for completing it – 18 months.

Leaders of the associations involved in the effort said they welcomed the department’s approach, though they differed on how much they saw it as a change of heart. Constantine (Deno) Curris, president of AASCU, the state-college group, described it as a “significant step” by the department and a “statement that the department is supportive of the concept of building a Voluntary System of Accountability” like the one AASCU and NASULGC are developing.

Schneider noted that college leaders “have been frustrated” that they have not gotten much credit from department leaders and the Spellings Commission for the work they have been doing for years on assessing student learning. And whether department officials intended it or not, the discussion generated by the Spellings Commission “was heard as a call for standardized testing, primarily if not exclusively.”

What’s important about the new grant, she said, is that it puts college officials and department leaders much more on the same page. “Everyone agrees that we have not been as transparent as we ought to be, as we might be,” she said. “The disagreement has not been about whether we should be accountable, but about making sure that we have forms of accountability that would actually strengthen learning,” for students and institutions. The cooperation between the government and college leaders makes that more likely, she said.

The extent to which the collaboration might represent a breakthrough might be most evident in the fact that even the National Association of Independent Colleges and Universities -- which has vigorously opposed department efforts to push colleges to use comparable, standardized measures of student learning -- viewed the moment favorably.

"The idea of putting a competitive grant out there through FIPSE is a very positive development," said Sarah A. Flanagan, vice president for government relations and policy development at the private college association. Flanagan said she had not seen the details of the proposal, so could not comment on every bit of its substance. But "we take this as a signal on [department officials'] part that they are willing to work in partnership with colleges in addressing some of the complex policy issues that face higher education."