Redoing Application Reading

COLUMBUS, Ohio -- Colleges and universities are preparing for a future in which student bodies are less white, Northeastern and Midwestern. That means changes for admissions readers, those who evaluate applications from prospective students.

Admissions officers from three institutions shed some light on those changes in a session Thursday at the National Association for College Admission Counseling’s 2016 national conference. One of those institutions, Pomona College, has already seen a radical shift in its student body, according to Associate Dean of Admissions Ashley Pallie.

“In 1984, at Pomona, we were only 7 percent underrepresented minorities, and that would have been black students, Hispanic, Latino and Native students,” Pallie said. “In 2015 that number was 27 percent. In the class that we just enrolled, so the first-year students who are on campus right now, just over 49 percent of them identified as domestic students of color. And so that includes Asian-American students, Hispanic, Latino, two or more races, Native students and black students, and then another 12 percent identify as international students.”

The shift has led to changes in the way Pomona thinks about diversity, Pallie said. For instance, it looked at how readers were evaluating applications from Asian-American students -- a major part of the population in California. Pomona is about to hold its fourth training since December on reviewing applications from Asian-Americans, looking at students’ backgrounds and patterns of movement.

Previously the college seemed to have viewed Asian-American students under three groups, according to Pallie. But it went back to the drawing board.

“We literally pulled up a map,” she said. “OK, so where’s China. Everyone knows China. That’s great. And then Japan, Korea, India. Can someone point out Cambodia and Malaysia and Laos?”

The college has also been talking about the way students in the different populations are described and whether that lines up with any culturally significant aspects of a community. Is it significant culturally, for example, if a student is described as quiet? Does that create the potential for bias? How is leadership defined in different communities? It’s also looked at immigration patterns and levels of educational attainment among different groups.

Pomona received 8,100 applications in 2015-16, admitting 9.5 percent. It has 14 full-time readers and four seasonal readers.

For a look at a larger institution, take Ohio State University. Ohio State received 49,389 applications in 2015-16, admitting 49 percent. It employs 77 full-time staff readers and 21 seasonal readers.

Ohio State emphasizes that its readers should look at a student’s entire application instead of just a few data points, said Vern Granger, associate vice president and director of enrollment services.

“We try to convey the message to them that our job is to build a class and not admit a class,” Granger said. “If we were just admitting a class, we wouldn’t be here. Because anybody can enter [grades] and SAT scores and admit however many students. Our job is to really build a class and get them to focus on those priorities.”

Readers must be educated about a university’s values and priorities, Granger said. An institution can want readers to evaluate an application with a holistic view, but the word “holistic” can be a buzzword that means different things to different people unless they have been educated about it.

Ongoing training, review and feedback are critical, Granger said. Ohio State focuses on its mission statement and strategic enrollment plan.

“The mission statement of your university is a foundation,” Granger said. “When you’re building your application review process and you’re working on training, you really need to be aware of the priorities of your institution.”

The Massachusetts Institute of Technology follows the idea that more data reduce the chance of bias in the application process, said Lauren Avalos, associate director of admissions. That means an MIT-specific application with five essay questions and an optional sixth essay to provide more information. MIT also tries to review applications in context -- for example, not penalizing students from backgrounds that often don’t have access to advanced placement courses.

“The value of more information is it gives us this opportunity to get a little closer to the student,” Avalos said.

MIT received 19,020 applications in 2015-16, admitting 7.9 percent. It employs 19 full-time readers and 11 seasonal readers.

Outside observation points to different types of biases in admissions. Michael Bastedo, a professor and director of the Center for the Study of Higher and Postsecondary Education at the University of Michigan, studied two unnamed flagship university admissions offices. He saw anchoring bias and correspondence bias, he said.

Anchoring bias is relying too heavily on the first piece of information someone sees when making decisions -- for example, an admissions reader looking over a sheet that has information on an applicant’s high school, and the information from that sheet then having a disproportionate impact on the reader’s evaluation of following data. Correspondence bias is tending to link facts to people’s personalities rather than the context surrounding them.

“When I did my fieldwork I saw people trying to fix these problems without identifying them the way I did,” Bastedo said. “They saw these problems and they were trying to train readers not to do them.”

Language monitoring was the first tactic Bastedo pointed out. So instead of talking about “a great essay” or “a poor essay,” admissions staff members would talk about “a helpful personal statement” or “a missed opportunity.” It might look at first glance like being politically correct, but it actually changed the way readers thought about applications, helping to keep them from overemphasizing one part or another, Bastedo said.

Another correction Bastedo suggested was reducing cognitive closure -- essentially making sure readers don’t try to force a decision early in the reading process. He also suggested changing error correction routines that had readers fearful of being told they had scored applications incorrectly.

“From the reader’s point of view, they live in fear of this phone call because they saw themselves making very high-stakes decisions,” he said. “They were being told they were messing up people’s lives.”

Bastedo suggested one more tactic to combat anchoring bias: considering the order of application information.