Let’s Have Better Debates About Standardized Tests

Reasonable people can come to different conclusions about the proper role of standardized testing, what qualities should be measured and how to measure them. Those debates are healthy and productive. It’s less productive when we stake out our positions without giving opposing arguments a fair hearing. Even worse are our tendencies to misrepresent opposing views.

Akil Bello’s recent piece “The Misguided War on Test Optional” illustrates this problem. Consider the way Bello misrepresents opposing views. Bello, who advocates for test-optional admission policies, describes one opponent as having published a “loving ode to elitism.” Supporters of standardized tests, in his account, “tend to look not at whom the tests hurt or what the tests miss but instead their sorting power.” Their core arguments against test-optional and other policies assume that “the purpose of college is to rank and sort members of society” and that the tools for doing so “should exist unquestioned in perpetuity.”

In my 30 years of experience in assessment and test preparation, I haven’t met people who hold the views Bello describes. Instead, I’ve found that people in this industry believe that to improve education, we need to take an honest look at measurable outcomes.

We often disagree about what should be on the tests, and how important the tests should be, but we tend to think that:

• The tests measure some important things. We believe this because of data we see on the predictive value of the tests, our examination of tests themselves, and/or our experience with people who take them.
• There are meaningful skill differences between people with very different scores.
• Standardized tests provide a common measure for people who went to different schools and had different academic experiences.
• The explanatory power of high school grades is diluted by grade inflation and differences between and within schools, classes and teachers.
• Other aspects of the application, such as essays and recommendations, deserve a place in the process, but are highly subjective and even more influenced by societal inequality than standardized tests are.

For these reasons and more, many of us believe that standardized tests deserve a place in the current admissions process. Reasonable people can disagree with any of the above conclusions, but we should be able to have that debate without misrepresenting opposing views. This particular tactic is part of the reason why our debates are so dysfunctional. Arguing against exaggerations of opposing views gets more attention and rallies our allies but it also alienates those who disagree while impeding common understanding and compromise.

We can do better. Let’s return to the principle that we should understand opposing views before forming an opinion. To that end, here’s a quick attempt at describing the views of those who want to abolish the testing requirements. They typically believe that the tests are unfair, irrelevant and magnify social inequality. They might believe that the tests measure the wrong things; others believe that no test could deserve a place in the process. Test opponents are concerned about demographic differences in test scores. For many of them, the differences themselves are conclusive evidence against the tests. They know plenty of people with great scores who didn’t succeed in life, and plenty who had terrible scores and achieved great success. And even if tests have some value, test critics question whether they are worth the costs involved—not just financial costs but also the time involved in preparing for the tests.

Once we acknowledge opposing views, we need to do more to bridge our divides. We also have to make good arguments and recognize the weaknesses of our own positions. Bello’s essay provides lessons there, though maybe not the ones he intended. For example:

• He argues that “all the benefits of testing continue to exist in a test-optional environment, though critics of the policy desperately want to pretend they do not.” This conclusion is based on the fact that at most colleges, applicants can and do submit scores, and therefore are able to take advantage of a good one. But this is misleading unless you know how colleges evaluate the decision not to submit scores. If people with low scores can bury them without it counting against them, then the difference between good scores and bad scores is diluted. If submitting grades were somehow optional, and people could hide their transcripts without penalty, wouldn’t it be fair to say that the advantages of good grades would be reduced? Moreover, the current test-optional approach makes it very difficult to tell whether a score would help you, which means that many students with scores that would help them don’t submit them, and many students who could earn scores that would help them don’t try their best because they think the tests don’t matter.
• Bello also asks, “Does creating winners and losers serve the country? Or does it merely perpetuate inequality and exclusion?” Are those the only two options? I’m pretty good at multiple choice tests, and I can think of other possibilities. Setting up false choices is a classic debate strategy that every citizen should know, and they should also know that false alternatives are debunked when one proposes reasonable alternatives. It’s inaccurate to describe the tests as “creating winners and losers,” as though the tests are arbitrarily anointing the chosen few. And the interrelationship between tests and inequality is, well, complicated. But there’s definitely more to it than choosing between the options Bello provides.
• Here’s my favorite: Bello claims that “the ire directed at the optional testing but not other optional elements of the application process should raise questions.” This tactic itself should raise questions. No matter what your opponents say, you can always cast doubt on their motives by saying that they should have been paying attention to something else, and their failure to do so raises disturbing questions about their priorities. Please watch out whenever anyone does this! It's perfectly legitimate for test professionals to make an argument within their expertise, even if they could have said something else.

There’s more to tackle in Bello’s essay, but I hope the above is sufficient to nudge the debate to a healthier place. But let’s not stop there. Often, supporters of tests are on the defensive, held to account for everything wrong in education. But let’s look at the other side. People don’t like tests, for reasons that are easy to understand, but what happens when they’re gone? Colleges still need to make admissions decisions, and getting rid of tests increases the influence of grades and other aspects of the application, and are we sure that’s a good thing? Would we be OK with making college more of an extension of high school, which is often an uninspiring trudge of compliance and punishment? Is it really better to force people to take tons of Advanced Placement courses that don’t interest them just to tread water with other applicants who have flawless GPAs? Shouldn’t there be a way for people who reject this unethical treadmill to demonstrate their skills?

And what happens when we bury relevant evidence of students’ inability to understand what they read and their inadequate math skills? Do those problems go away? Of course not. When students are in over their heads, they fail out, drop out, pick easier majors, and generally have a miserable time in college. It’s easy to feel sympathy for the people who don’t get into their target colleges because their test scores are too low, but let’s also have sympathy for the people who think they are ready for college but aren’t. Let’s consider the 40 million people who left college without a degree, and the high percentage of those people who are behind on their student loan payments. While many factors contribute to this unacceptable failure rate, the high percentage of high school graduates who are unprepared for college is a part of the story we shouldn’t ignore.

Keep in mind that while our high college dropout rates are a serious problem, there are also consequences when unprepared students stay in college and acquire degrees. Colleges are measured in part by graduation rates. Professors are measured in part by student evaluations, and student evaluations are highly dependent on expected grades, resulting in unbelievable pressure to give OK students great grades and unskilled students passing grades. Everyone in the educational chain has an incentive to push these students along, and that’s why we need a more objective measure of what students are actually learning.

Denying reality catches up to you eventually. There’s a reason why so many people who don’t want to go to college spend years of their lives and untold thousands of dollars to acquire this credential: nobody trusts high school diplomas. And now, grade inflation and legitimate questions about the value of college are eroding the perceived importance of college degrees, which forces people to acquire graduate degrees just to be competitive in a global job market. We’re caught in a prisoner’s dilemma: No one would design this system from scratch, but individual incentives push us to create a system that’s worse for everyone.

For these reasons and more, standardized tests provide a needed counterweight to negative trends in education. I'd be thrilled if high school education were good enough to be a sufficient indicator of who is ready for college and who can handle the rigor of our most challenging courses. But that’s not where we are now, and hiding evidence of our problems only makes them worse. Making progress here depends on cooperation from all kinds of people, and turning the debate over standardized testing into another example of our dysfunctional debate culture works against this goal.