Assessment: It's the Law

If teaching a 300-plus person course isn't enough work, faculty members who do so at Iowa's three public universities have new duties starting this fall. By state law, they must create and use "formative and summative assessments" and submit a plan for using those assessments to improve student learning.

But don’t call them accountability measures, said Herman Quirmbach, a state senator who is also a professor of economics at Iowa State University.

“This is about faculty being engaged in student learning,” said Quirmbach, who prefers the plan’s “continuous improvement” tag, as articulated by the Board of Regents in its strategic plan and reiterated by the state legislation passed in 2012. They take effect this fall and require universities overseen by the Board of Regents – Iowa State, the University of Iowa and Northern Iowa University – to formally monitor student learning at the course level and prepare annual reports on those courses for the Board of Regents. The board will report to back to the legislature.

“This is not about accountability,” Quirmbach, chair of the state senate's Education Committee, continued. “This is not about devaluing the faculty, and if it goes in that direction it will be counterproductive.”

Others, including Joe Gorton, professor of criminology at Northern Iowa, aren't convinced. "Anyone who believes that this kind of bureaucratic micromanagement is going to improve higher education in Iowa immediately categorizes themselves as someone who does not have the first clue about the linkage between university pedagogy and student outcomes," he in an e-mail. "This is time and energy taken away from our core missions of teaching and research."

The public would be "shocked if it knew how much wasted time Iowa’s professors have to spend on these kinds of bureaucratic mandates," added Gorton, who serves as president of the local United Faculty, a union affiliated with the American Association of University Professors.

Quirmbach said the continuous improvement plan is an extension of longtime gathering of data at the university and program level to monitor student learning. In recent years, however, he and other legislators have pushed to monitor student learning at the course level, “where the rubber meets the road.” It's part of Iowa's commitment to higher education, he said, noting its three research universities, despite its small population of 3 million people.

“For each undergraduate course the institution shall collect and use the results of formative and summative assessments in its continuous improvement plan,” the legislation reads. “The board shall annually evaluate the effectiveness of the plans and shall submit an executive summary of its findings and recommendations in its annual strategic plan progress report, a copy of which shall be submitted to the General Assembly."

The new requirement applies to courses -- not just sections -- with 300 or more students enrolled annually during the first year; 200 or more the second; and 100 or more in the third and final year of the plan's rollout. Because there are some costs associated with reporting mandates, Quirmbach said the legislature thought it would get the most “bang for its buck” by focusing on large courses.

Beyond a timeline for implementation and the "summative and formative" assessment stipulations, the plan is somewhat vague. That's in part to afford faculty flexibility to define their own learning outcomes and methods of measurement, whether by tests, written assignments or other means; professors -- subject matter experts -- know best what their students should be learning, Quirmbach said.

Some universities and programs are further along than others. Professors in accredited programs throughout the state, such as engineering, already monitor student learning formally by course. They are free to use those assessments to meet the new requirement.  And at the University of Iowa, some departments already require internal reporting from professors, said Erika Lawrence, professor of psychology and Faculty Senate chair. In her department, short, content-based multiple-choice assessments are given out at the beginning and end of semesters to measure learning. Those, too, will figure into meeting the new requirement, she said. Consequently, there hasn't been too much of a negative reaction to the plan.

Not so during a departmental faculty meeting at Northern Iowa, where the plan was recently explained and "most perceived it as busywork," said Mary DeSoto, professor of psychology. 

"Most all faculty regularly do things like switch to current text books every few years, modify lectures based on current research they become aware of, make various alterations in courses based on experiences," she said in an e-mail. "Documenting it (writing a report) is probably not seen as helpful by most. I do not at all doubt that it was passed with the very best of intentions, but in my view it won't probably actually have the effect intended."

Template assessment tools are being drafted by at least two of the universities to help professors new to formal reporting. Northern Iowa's template, for example, will provide space for professors to record their assessment strategy, what was learned from the assessment, and what will be changed as a result, a university spokesman said. All reports must focus on student learning outcomes and objectives as outlined in course syllabuses, and must include some kind of data collection on those outcomes. Evidence of changes to the course based on gathered data is also required, as is evidence of continuous reassessment.

At Iowa State, more than 560 courses will be affected by the end of the plan’s third year. This fall, 167 courses, mostly in the College of Liberal Arts and Sciences, will be impacted. At the University of Iowa, 145 courses will be impacted this year, with 545 total courses impacted by the end of 2016. At the University of Northern Iowa, the plan will affect about 60 courses in the first year.

Planning for next semester’s new requirement is under way now at all three universities, with most taking a “train the trainer” approaching at the college or department level.

Karen Zunkel, director for undergraduate programs and academic quality in the provost's office at Iowa State, which will be most impacted by the plan, said faculty reaction has been mixed. Some professors have asked if the information gathered will be used for tenure and promotion decisions, she said. But she’s assured them that it’s not the goal of the legislation, and that information will be collected by course, not by instructor.

Similar to Gorton, other professors have said they’re already monitoring student learning – the job of a professional educator – and that the reporting is just more paperwork, Zunkel said. But there’s also feeling among faculty that “if it’s legislated and we need to do it, let’s make it something that’s useful to us.”

One of the most appealing aspects of the plan so far, she said, is that it will require instructors teaching different sections of the same course to meet, discuss and record what they want their students to learn and how they’ll measure it. (In courses with 300 students divided into sections taught by multiple instructors, all will monitor student learning. But only one professor will combine their data and prepare a course report to be passed on to the Board of Regents.)

“We need to get together and have those discussions at the course level,” Zunkel said. Professors won't be expected to "overhaul" their courses the first year, she added, but rather focus on measuring and improving on select goals.

Veronica Dark, instructor for a multisection introductory psychology course and incoming Faculty Senate chair at Iowa State, also highlighted discussions across sections as a benefit of the legislation.

“We as a Psychology 101 team don’t do that enough,” she said, adding that in planning for continuous improvement, the team discovered that not all instructors were listing course outcomes and objectives in their syllabuses. “Now we will.”

Still, Dark said, some faculty feel they’ve got a “good product” in their courses, and don’t need any outside mandate for improvement – particularly “continuous improvement.”

“That label suggests you’ve got to keep changing constantly,” she said. “If we ‘ain’t broke,’ why fix us? I think that’s the resistance you see.”

But once faculty see that the plan is flexible and that they’ll decide what they monitor, she said, “I think it will make us think a little bit more about what we do and that in itself is a good outcome.”

As for fears that information gathered will eventually be used against faculty for personnel decisions or other potentially punitive means, Dark said faculty are reporting on their courses in “good faith” that that won’t be the case.

Sheila Doyle Koppin, spokeswoman for the Board of Regents, said the board has a “longstanding commitment to specifying and measuring expected learning outcomes for all undergraduate programs.” The new requirements are not meant to hinder faculty from teaching, she said, but rather “support and enhance” that ability.

Gorton disagreed. Between other required reporting at the department level and above, strategic plan planning and execution and academic programs reviews, there's increasingly little time to teach, he said. And the problem isn't unique to Iowa.

"Here and throughout the country, universities and colleges are being transformed in ways that make us less efficient financially and less effective in our responses to the critically important education and research needs facing our nation," he said.