Are We Sure We’re Better Off Without Standardized Tests?

Critics of standardized tests won a big victory recently, as the American Bar Association came much closer to allowing law schools to go test optional. It’s not over yet: the final decision hasn’t been made, and even if law schools can go test optional, they won’t have to. A quick survey by Kaplan Test Prep, where I used to work, indicated that schools are likely to keep their testing requirements. And 60 law deans wrote to the ABA council expressing deep concerns about the move. Surely that indicates some support in the law school community for testing requirements. Still, the trend is clear, and the final outcome is easy to predict: sooner or later, probably sooner, law schools will be able to go test optional.

For people who don’t like standardized tests, which in practice is almost everybody, this seems like a victory. Applicants who lack the skills these tests measure have special reason to celebrate. For them, the LSAT seems like an arbitrary barrier between them and their career goals. But is it? Are we sure that the law school admissions process (or other admissions processes) are better without standardized tests? I don’t think so, both because of the value that they offer and because of the gap they leave when they’re absent.

Relevance of LSAT Scores

In these pages in June, I made the case for the value of the LSAT. I’m sticking by that argument. The LSAT is well designed, measures relevant skills and predicts performance not only in law school but on the bar exam. In the test-prep community, the LSAT is recognized and respected for performing its function exceptionally well. There’s a reason that it’s remained very stable for the last 30 years, and those of us who have taught LSAT prep know well that the reasoning skills it measures are essential to making sound logical decisions, which seems somewhat relevant in the study and practice of law.

And if you think the LSAT is irrelevant, I ask, would you want someone representing you who scored at the bottom of a test of logical reasoning and reading comprehension? If, all else being equal, you would prefer the higher scorer, then to some extent you’re in favor of the LSAT.

While I’d be thrilled (really) to explain what the LSAT tests and why those skills are so important, I’d like to focus here on the consequences of eliminating standardized tests. If they’re gone, and no other factors are added, then every other factor could become more important, which magnifies their strengths and their weaknesses.

Grades

Consider grades, which are typically the No. 1 factor for admission to law school, college and most other programs. Grades deserve to be part of the picture, but how confident are you that applicants with good grades have what it takes to succeed at the next level? Doesn’t it depend on what was required to get those grades?

We’d like to think that high GPAs would guarantee strong critical thinking and reading comprehension skills, but that would be naïve in today’s environment, where reading skills are de-emphasized and critical thinking is often absent. From my time developing assessments for Pearson Higher Ed, I can confidently report that the overwhelming focus was on basic knowledge and comprehension. Attempts to climb the ladder of higher-order skills were met with opposition from basically every stakeholder, from student to professors to publishers.

That doesn’t mean that college is easy: I’ve been amazed to learn that college is extending the high school practice of loading down students with tons of low-stakes busywork assignments. The people who succeed in that environment demonstrate persistence and focus. But reasoning logically is an optional extra.

It’s hard to blame the professors for this situation: I consistently hear that the students themselves are demanding this kind of approach, which matches their high school experience, where they excelled. We might like professors to hold the line and demand that students demonstrate higher-order skills, but professors who do that court student revolts that could ruin their careers. The prudent move is to move with the times.

Grade Inflation

No matter what you think about grades, you have to admit that their value is undermined by grade inflation, which is real. If almost everyone has great grades, then grades aren’t much of a differentiator. In that case, great grades don’t get you in, but slight missteps keep you out. It’s the educational equivalent of Uber ratings: almost everyone is close to perfect, and so the ratings tell you nothing, but just a few very negative reviews can outweigh tons of positive experiences.

Or maybe the better example is the National Basketball Association contest, where the only possible scores are six, seven, eight, nine and 10, and in practice every good dunk gets either a nine or a 10. Sure, it’s fun to wave a sign with a 10, but if you’re waving it all the time, what’s the point?

We already see the effects of grade inflation in high school, where A averages are so common that ambitious students are forced to take tons of time-consuming Advanced Placement courses that don’t interest them at all, simply because “colleges like them.” Extending this dynamic to graduate admissions would not be an improvement.

Alternate Paths

Now consider the applicants who don’t have good grades. Maybe they got it together a little later in life, or took harder classes, or maybe they rebelled against an uninspiring educational program of compliance and punishments. If grades are king and standardized tests are out of the picture, then these people have little opportunity to provide objective evidence that they’ll succeed at the next level. Right now, those applicants can prove their skills by taking the same test that everyone else takes. Without standardized tests, though, we’re locking in the results of a system that deserves deep skepticism.

Other Factors

But what about other factors in admissions, such as essays, recommendations, work history, internships and so on? Would the admissions process be better if those factors had greater weight? I’m thinking no. First of all, they’re highly subjective. Reasonable people will have very different opinions about admissions essays, and it’s virtually impossible to be fair and consistent when comparing every essay in a big stack.

Recommendation letters are also “unfair” in that people have different recommenders. Let’s assume that a recommendation says something insightful about an applicant. (A lot of them don’t). Who is to say what that recommender would have said about the other applicants? There’s no way to know. Job experiences and internships have the same issues. Yes, they can add some flavor and round out an application, but evaluating them is prone to bias, and not everyone has the same opportunities to land those gigs.

Economic Inequality

If economic inequality is important to you, as it should be for all of us, then you have reason to be concerned about standardized tests. It’s a real issue, and those of us in the test-prep business have an obligation to offer low-cost, effective options. But if you think access to awesome test-prep tutors (like me!) is a concern, don’t you have greater concerns about admissions consulting? While many admissions consultants are fantastic people who care about students and draw lines when it comes to essay editing, others have fewer scruples.

In test prep, people like me can help students build the skills that the tests measure. But it’s not automatic: the student puts in the work and actually takes the test. With admissions essays, there’s no way to know how much help the student got, and so placing more emphasis on them will make inequality worse. Ditto for letters of recommendation, internships and job history. Emphasizing those will only magnify inequality. Who do you think has the access to get those prestigious (but unpaid) internships and awesome (but expensive) summer programs? Not the students who need to take on minimum-wage jobs during high school and college just to survive, that’s for sure.

There are no easy remedies for economic inequality that permeates society. But that doesn’t mean that every option is equally bad, and there’s a good case that standardized tests, on balance, give everyone a chance to compete on the same playing field. And at least we know who actually took the test.

One argument against standardized tests such as the LSAT is that they produce racial disparities. And they do. But different results, by themselves, do not prove that we should eliminate the test. After all, if we wanted to ensure equal outcomes, we could assign admissions spots entirely by lottery, regardless of qualifications. But we don’t do that, because qualifications matter. If a test measures relevant skills but produces disparate results, the solution is not to toss the test. Rather, those test results should be seen as a guide helping us determine where help is needed.

If you’re convinced that the tests are pointless for everyone, then the racial disparity data makes it even more important to eliminate them. But it’s a different story if you agree with me that they add valuable information to the admission process. If that’s true, eliminating the test would be like pulling out the batteries of your smoke detector. The noise may be annoying, but it’s trying to tell you something important.

Market Failure

Free markets can be great, and we’ve seen the benefits of allowing people to exchange goods and services without excessive regulations. But we’ve also seen the consequences of abandoning standards to give people what they say they want.

Financial regulators, building inspectors and standardized test professionals may not be the most popular people at parties, but what happens when they’re absent? You get financial meltdowns, building collapses and the modern higher education system, which forces millions of students to spend years of their lives and small fortunes trying to prove that they have the skills that should have been required to graduate from high school.

Higher education is supposed to provide a check on escalating credentialing, but the incentives too often go the other way. What happens when:

• Colleges and universities are rewarded for attracting tons of applicants, even when those applicants aren’t qualified?
• Those same institutions are punished when students don’t graduate?
• Test-optional policies boost application numbers, as more and more applicants look competitive when their weaknesses stay hidden?
• Instructors are dependent on student evaluations, which are highly depending on expectations of high grades?
• There are few, if any, objective measurements of what most students actually learn?

The consequences are easy to predict and observe: educational credentials will mean less. If there are no alternative paths to acquire credentials, more people will be forced to go to college and grad school, even when doing so means less than it used to. In the long run, a system that forces more and more people to acquire credentials that mean less and less will be disrupted. There’s too much inefficiency for this to continue forever, and alternatives to traditional education will emerge.

For-profit higher education was supposed to be the disruptive force we needed, but the results so far have been less than encouraging. In my experience working with quite a few of them, I’ve found that market incentives push them to slash costs, reduce or eliminate admissions standards, and prioritize graduation rates over rigor. Going against these incentives would be laudable, but we can’t expect people to choose less profitable paths on their own. Access is great, but access minus standards equals disaster. At some stage, we need sober and dispassionate regulations, if only to save us from ourselves. In education, that’s a strong reason to keep standardized tests.

A Final Plea

Even the defenders of standardized test admit that they’re imperfect and generally unpopular. Those of us in the test-prep industry know from personal experience that most people (even those who do well on tests) would prefer that they all go away forever. We get it. But there are still good reasons to have them, both for what standardized tests offer and what happens when they’re not around. This is harder case to make with the SAT, which is a deeply flawed instrument. Still, we have the potential to develop better standardized tests. And with the LSAT, we already have a very good one. It would be a shame to lose it.