SEO Headline (Max 60 characters)

Arbitrating the Use of Student Evaluations of Teaching

August 31, 2018

An arbitrator ordered Ryerson University in Canada to amend its faculty collective bargaining agreement to ensure that student evaluations of teaching (SETs) are not used to measure teaching effectiveness for promotion or tenure. The evaluations’ numerical weighting system also should be replaced with an alphabetical one, according to the order, and both Ryerson administrators and the campus Faculty Association must meet to agree upon “appropriate, user-friendly, intelligible and easily accessible mode of presentation of [evaluation] data in the form of a frequency distribution together with response rates.”

The order is the result of an ongoing dispute between Ryerson and its faculty association over the use of Faculty Course Surveys, its SETs, in personnel decisions such as tenure and promotion. Much research suggests that SETs reflect student biases and other factors unrelated to one’s teaching, and that these evaluations therefore should not be used in high-stakes personnel decisions. The Canadian arbitrator’s order cites this evidence, saying "the best way to assess teaching effectiveness is through the careful assessment of the teaching dossier and in-class peer evaluations.”

 While it is “probably impossible to precisely measure teaching effectiveness,” the order says, “the difficulties in doing so cannot serve as a justification for over-relying on a tool ­– the SET – that the evidence indicates generates ratings but has little usefulness in measuring teaching effectiveness.”

At the same time, evaluation results “can continue to be used in tenure and promotion, when the results are presented as frequency distributions, and when the end users are appropriately educated and cautioned about the inherent limitations both about the tool and the information it generates.” SET results “provide information about the student experience, and, contextualized, are appropriately considered for tenure and promotion although, to repeat, not for reaching conclusions about teaching effectiveness,” reads the order.

Deans and those who otherwise evaluate faculty members also must be educated on inherent biases present within students’ evaluations of teaching and Ryerson and its faculty should establish a committee to study the current evaluation tool and possible improvements. And until both parties agree on a substitution for the online evaluation system for non-online courses, it is to be discontinued for probationary faculty, the order says.

Share Article

Colleen Flaherty

Colleen Flaherty, Reporter, covers faculty issues for Inside Higher Ed. Prior to joining the publication in 2012, Colleen was military editor at the Killeen Daily Herald, outside Fort Hood, Texas. Before that, she covered government and land use issues for the Greenwich Time and Hersam Acorn Newspapers in her home state of Connecticut. After graduating from McGill University in Montreal in 2005 with a degree in English literature, Colleen taught English and English as a second language in public schools in the Bronx, N.Y. She earned her M.S.Ed. from City University of New York Lehman College in 2008 as part of the New York City Teaching Fellows program. 

Back to Top