Bias and the Application Process

The academy, particularly in its upper echelons, is severely lacking on the diversity front. At many levels, including admissions, funding and even hiring, one of the major gateways to advancement is the application process. When I agreed to serve as an application reviewer for a fellowship for the first time this year, other than the usual tinge of impostor syndrome, I felt confident that I knew what I was in for. After all, I had read about the applications process, and I have been coached on my own applications, several of which were successful.

Still, I was not prepared for what it was actually like to be on a team that reviewed hundreds of applications. At every turn, I could see how the application review process disadvantages applicants from underrepresented groups. As a social-justice-minded queer academic, I feel compelled to share a list of my observations and some thoughts on how those of us reviewing applications and those of us applying can push back against these systemic inequalities in the application process.

How Reviewing Happens

Wow, do reviewers skim! For round one, we were told to spend between one and five minutes reviewing each 30-page application. That’s 10 seconds or fewer per page of short essays, check boxes, recommendation letters, CVs, etc. Your favorite sentence? That subtle wording? I probably did not see them as I scrolled by, and I took about 15 minutes total per application. If the one sentence that I happened to read in that paragraph was self-doubting, or if I was unsure how it answers the question, “Why should I invite you to interview?” then it was hard to give the application a positive review.

Applicants, the reasons you are great -- and, yes, there are reasons you are great -- have to be so simple and obvious that reviewers pick up on them in a matter of seconds. Reviewers, one of the things that I loved about this particular process is that it forced me to find and write down at least one exceptionally good thing from every application. That really pushed against the perils of skimming, as well as my implicit biases, at the same time. If you are designing a review process, this is a small but important structural addition you can make. If you are reviewing, you can make doing this a personal requirement even if it is not formally required.

Reviewers seek out reasons to quickly reject. We have a huge stack of applications to review, and we can only interview a tiny fraction of people, often less than 10 percent of all applicants. Once we spot a flaw in the application, it can easily feel as though we have done our job and narrowed down the pool of applicants. It is extremely tempting to quickly give a low rating and move on to the next application after finding a weakness. That has deleterious effects on minority and women applicants, who are often socialized to hedge expressions of their strengths by pointing out their own weaknesses.

As applicants, it can help to know that reviewers are already looking for flaws in order to use them against us. As reviewers, we can help by sticking to predetermined evaluation criteria instead of leveraging the feelings of inadequacy underrepresented applicants are socialized to express against them.

One bad review dooms an application. Three reviewers skimmed each application, and their average ratings determined who never got a closer look. It is a safe bet that at least 10 percent of applications will get positive reviews from all three reviewers. Thus, one bad review is often enough to reject an application without an interview.

As applicants, we should not only write an application that most reviewers will like. We also need to write an application that none of the reviewers will dislike. Personally, I find it helpful (although painful) to read through my writing sentence by sentence and, for each one, ask myself, “What is the worst way this could be interpreted?” and, “Could this sentence give readers a reason to reject me or my argument?” As reviewers, before we give a low rating in one or more areas, it is worth asking if those areas are really bad enough to dismiss the applicant outright, because that is often the effect we have. Meanwhile, more privileged applicants often seem to have no particular weaknesses and glide through to the next phase, despite sometimes being less desirable candidates overall.

Inside the Applications

Male applicants tend to oversell ourselves. Those from high-status universities particularly do that, while female applicants tend to undersell themselves. There are social reasons behind this -- real pressures not to “lean in.” Application reviews, however, favor privileged rhetoric that boldly states the applicant’s strengths and omits negative self-evaluations. The survival strategies underrepresented applicants have learned in other spaces may work against them here. For instance, minority applicants with ample experience in an area often described it as “limited” or something that they “need to improve.”

Reviewers and applicants both should avoid applicants’ own feelings about their qualifications. Applicants just do not have enough information to accurately judge their own relative strengths, because they cannot see the other applications or reviewer criteria. On top of this, socialization means self-evaluations will be biased in favor of privileged applicants. When writing and reviewing applications, we can reduce the effect of this confidence gap (read: structural inequality) by sticking to other measures. For example, how long have they worked with a method? How good is their understanding of a problem?

Give friendly reviewers something to work with. This can be tricky. Moreover, coming out about our identities and struggles in an application (where sometimes even race, ethnicity and gender may not be obvious to reviewers) risks discrimination. Still, a reviewer who wants to offset the challenges to underrepresented groups in STEM fields cannot do that unless they know about the identities their applicants have. Similarly, many of us have held nonacademic jobs to make ends meet. Often that means we have less time for building our CVs than applicants with more academic or personal funding. Mentioning this lets a friendly reviewer emphasize the quality of research experience over the quantity.

Honest recommendations doom applications. Almost every letter says the applicant is perfect, or maybe that they are just near perfect in one area but perfect everywhere else. Most letter writers know this. Unlike grade inflation, it is not even debated. Because any honest review admits a candidate is not perfect, an honest review puts a candidate at the bottom of a thick pile of “perfect” applications. Structurally, the letter writers least likely to know and follow this custom are those on the periphery of academe.

Consequently, applicants at the margins are disproportionately likely to receive these letters. Reviewers can be conscious of this when comparing typical reviews to more honest ones.

Bonus: How Reviewers Can Spot Privileged Applicants

Often, applicants specify a problem they would like to work on -- usually one that others have been working on for years. When the applicant describes such a problem as trivial, it is a sure sign of an applicant with some problematic privilege. Minority applicants usually get points because they respect other people’s work and the challenges they face.

For applications with a social justice component, “I volunteered at a soup kitchen once and I liked the feeling when the patrons smiled at me” is a sure sign that an applicant does not get it. Not everyone has the opportunity to be involved in long-term work for systemic change, but I am skeptical of any applicant who talks about social justice and does not display understanding of the nature of the struggle.

People have proposed numerous structural ways to address inequality in higher education. Applications and reviews are not a complete solution, but they can be important sites of change. Have you experienced other ways applications processes can re-entrench bias? Do you have other tips on pushing back? Share them in the comments!