Accent on Bias

Rate My Professors is a student evaluation site that frustrates many professors, who say that the nonscientific standards leave faculty members open to unfair ratings.

Last month, a study documented the extent to which students use different sets of words (many of them with gender implications) to discuss their male and female professors. Now a new study looks at how students on Rate My Professors rate instructors who have Asian-sounding last names, and the results suggest that these instructors are getting significantly lower scores than those with other last names in Rate My Professors' categories of clarity and helpfulness.

The author of the study, who also examined comments students make about the instructors, said that his findings raise questions about whether American colleges and universities are as international in outlook at they boast of being -- and whether Asian instructors are being reviewed fairly. The study -- "She Does Have an Accent But" -- has just been published in the journal Language in Society (abstract available here).

Nicholas Close Subtirelu, a doctoral student in applied linguistics at Georgia State University, wrote the paper based on an examination of Rate My Professors ratings given to more than 1,000 mathematics professors with Chinese- or Korean-sounding last names and comparing these results to a much larger sample of instructors whose last names did not suggest an Asian background. (Subtirelu acknowledges that this approach will result in some instructors with Asian names being grouped in the wrong group, and vice versa, and that those with Asian names need not be recent immigrants or foreign citizens.) Many of those with Asian names appear to be employed as teaching assistants, but this is not true of all of them.

Over all, he found that instructors with "American" last names received clarity scores that were 0.60 to 0.80 points higher than did those with Asian names (on a five-point scale), and that they received scores 0.16 to 0.40 points higher on the helpfulness scale.

He broke out his findings by whether the instructor was male or female, and looked at region of the United States.

Consider the results for mean clarity scores.

Mean Scores on Clarity for Mathematics Instructors

While the students rated their instructors with non-Asian last names consistently higher than those with Asian last names, the gaps were the largest in the South. Rankings of instructors with Asian male last names in the West were the closest to those with other names.

Subtirelu then looked at the comments students post, and noticed a pattern in which many students introduce remarks (frequently praise) by saying than an instructor "has an accent, but." He also noted cases where students remarked positively about language, but seemed to view that as a surprise or as something that needed to be shared about someone with an Asian last name. "Her English is perfect" is an example of such a comment. These comments -- seemingly positive -- suggest a focus of students on Asian instructors' language skills in evaluating them.

The findings suggest to Subtirelu that there are serious issues facing American colleges and universities that rely on Asian teaching assistants, but his view of the issues is not that American students are being poorly taught. He acknowledges that there may be some individual instructors who are difficult to understand because of their accents or speaking abilities. But Subtirelu is working on another research project in which he's sitting in on classes led by Asian instructors.

Many have accents, he said. But they are not actually difficult to understand if one makes a little effort. He said that he is concerned that simply having an accent is being viewed as negative. Students appear to be "pushing back against this extra labor of interacting with their instructor, to overcome this extra difficulty that they face with someone who doesn't share their background." He added that "this is a big problem for an institution that wants to be an international university."

Subtirelu said that students should not be viewing their Asian instructors entirely through a prism of an unfamiliar accent. He said he worried that the references to accents might lead some students to "avoidance behavior," where they try to sign up for other teaching assistants. "This is very concerning."

He said his skepticism of the idea that Asian professors weren't understandable came both from his sitting in on many classes and his knowledge of hiring practices and training for teaching assistants. It was once true that many American universities just threw new T.A.s (foreign born or not) into classrooms. But in the last 30 years, American universities have imposed more testing of English language skills on foreign graduate students and offered much more training.

"Given all of that, I'm really skeptical of the idea that they lack English proficiency, that they are unintelligible," Subtirelu said. But the question is about whether a little effort is needed "on both sides" for better comprehension. He noted that in his class observations of Asian and non-Asian instructors, he will sometimes miss a word, and he guesses many students do as well. But many times no one asks. Subtirelu said that his findings make him wonder if "everyone is paying attention and doing what they need to do to adapt" to a new accent, rather than just focusing on the accent as a potential problem.

He also noted that Americans love some foreign accents, even if they too may sometimes make words hard to understand. "There's certainly a hierarchy of accents in the world, and British accents tend to come out on top," he said.

If you are wondering, Subtirelu grew up in Ohio and, to this reporter in a phone conversation, has no discernible accent.