Ethical College Admissions: Pro-Test or Protest

Not long ago I saw a television commercial extolling the virtues of American energy independence. As someone who spent the summer of 1979 sitting in lines waiting for gasoline, and as someone who suspects that the war in Iraq may have been as much about oil as weapons of mass destruction, energy independence seems like something no one could oppose.

Of course, what is important in advertising is not necessarily what is said but rather what is left unsaid. At the end of the energy independence ad what piqued my attention was fine print stating it was sponsored by America’s oil and gas industry. It turns out that “energy independence” is a product of and a euphemism for fracking, a practice not as universally beloved as energy independence.

At least the oil and gas industry is making its case in what is clearly an advertisement. What is more objectionable is advocacy masquerading as research or journalism.

In 2016 an article in the medical journal JAMA Internal Medicine revealed that a seminal 1967 study by three Harvard professors on the causes of heart disease concluding that saturated fat rather than sugar was the primary culprit was secretly paid for by the Sugar Research Foundation. That study shaped five decades of public policy on health and diet. It is possible that the conclusions of the research were not influenced by the funding, but the sugar industry’s involvement is at best suspicious and at worst a genuine example of “fake news.”

The Johns Hopkins University Press has just released a new book examining the use of standardized testing in college admission, Measuring Success: Testing, Grades, and the Future of College Admissions. While that doesn’t sound like something I would include in my beach reading this summer, it’s potentially an important contribution to the literature of college admission.

I say “potentially” because there are certainly skeptics who wonder if the new book will turn out to be a balanced study of the issues related to standardized testing or pro-testing propaganda. The skepticism derives partly from the fact that two of the three editors of the new volume are employees of the College Board, while the third, who recently left his position as senior vice president at the College Board for a similar post at the American Institutes for Research, led the redesign of the new SAT.

The other cause for skepticism is an article that appeared last fall on the website of The Atlantic Monthly as “sponsored content,” a nicer description than “infomercial” or “advertorial.” The sponsor of the article, “When Grades Don’t Show the Whole Picture,” was none other than the College Board. That article cites research from the book and, based on the book’s description on the Johns Hopkins Press website and in a Chronicle of Higher Education article, appears to be a preview of the argument laid out in the book.

The Atlantic Monthly advertorial lays out a two-legged argument, both of which lead to the inevitable conclusion that standardized testing, and in particular the new and improved SAT, is an essential tool for admissions officers and for students.

The first leg of the argument is that grade inflation in high schools makes it harder for admissions officers to “fairly differentiate” among students applying to college. It quotes research from Michael Hurwitz of the College Board and Jason Lee from the University of Georgia that high school grade point averages are higher than ever.

Over the past 20 years the number of students with an A average taking the SAT has increased from 39 percent to 47 percent, an increase they ascribe to grade inflation. I don’t quarrel with the conclusion that there is grade inflation, but I wonder how meaningful that statistic is. It is based on data self-reported by students on the questionnaire they may complete when they register for the SAT.

But how reliable is self-reported data? As a teacher I long ago learned that many students are overly optimistic about their grade point averages. They suffer from selective memory, remembering the one A-minus grade they received on a quiz and forgetting the 10 lower grades on major assignments.

Just ask U.S. News about the dangers of self-reported data. Last week it removed Temple University's Online M.B.A. program from the rankings based on concerns about incorrect self-reported data, data U.S. News apparently doesn’t vet.

While we’re on the subject of inflation, what about score inflation? When the College Board introduced the new SAT two years ago, it argued that it was a different test and that scores from the old and new SAT shouldn’t be compared. If that was the case, why keep the 200-800 scale? Is that part of the brand?

The other day I had a conversation with one of my seniors, having to break it to him that his 1400 SAT score might not be as impressive as he hopes for his first-choice college. At that school 1400 on the old SAT was at the 75th percentile. On the new SAT, a score of 1400 is at the 50th percentile.

The other leg of the argument is that the new SAT is an engine of opportunity for the very students standardized testing has previously been criticized as disadvantaging -- students of color, first-generation students and those from low-income households. It’s a fascinating -- and convoluted -- attempt at persuasion.

It begins by arguing that grade inflation is an instrument of privilege. The Hurwitz and Lee research suggests that grade inflation is disproportionate, three times more likely to occur at private high schools than public or charter schools.

The second part of the argument is an attempt to discredit the test-optional movement. The College Board argues that test-optional policies receive more media attention than warranted. The advertorial argues that test-optional policies do not produce greater socioeconomic and racial diversity. Finally, it concludes that the SAT is an important part of a holistic admission process, a claim that sounds analogous to the advertising claim that sugary children’s cereals are part of a balanced breakfast.

But, to borrow from Shakespeare, doth the College Board protest too much? It is certainly true that increasing diversity has been a justification for test-optional policies. Far more institutions have chosen to go test optional because they don’t find that test scores provide any added value to grades, even inflated grades, in making admission decisions.

So if neither high school grades nor test-optional policies serve as engines of opportunity, is there a tool to come to the rescue of both students and colleges? If you didn’t say “the new SAT,” guess again.

How does this happen? First, the new SAT is argued to measure not aptitude but what students have learned in school. Second, the College Board’s partnership with Khan Academy allows students to practice (as opposed to prep) to improve their SAT scores and their chances of admission.

It is here that the two-legged argument tips over. The argument against grades is that grades without context (grade inflation) are misleading. But isn’t that true of scores as well? We know that we shouldn’t consider two students with identical scores identical if one has grown up in an affluent family and the other attended an inner-city or rural school without a college-going culture. But doesn’t that hold true for practice/prep as well? If one student has a score of 1200 based on hours of Khan Academy practice, and another student has a 1200 with no practice, isn’t that important context? Should those scores be evaluated as equal?

I always worry that movie trailers are misleading, containing the very best moments from a film. In the case of the new book, I hope there’s more than what appeared in The Atlantic “sponsored content” trailer.