Grading the New SAT

The College Board on Wednesday announced major changes to the SAT, including a substantial revision to the writing test that was added in 2005 in the last major overhaul of the admissions test.

A number of the changes appear designed to respond to the growing chorus of criticism of the SAT. And the announcement is in some ways surprising for the extent to which it admits that some past changes didn't work. For example, the College Board news release on the changes notes that the writing test added in 2005 "has not contributed significantly to the overall predictive power of the exam."

Given that the SAT is designed to predict college success, and that the writing test was the most prominent change of the 2005 revisions, that's a fairly dramatic statement.

Among the changes announced Wednesday (and scheduled to take place in 2016):

• The current writing test, in which students cite their own experiences or values to respond to a statement, will be replaced with one in which students respond to a passage of writing, and must analyze evidence. The students will be evaluated on both their analysis and their writing.
• The writing test will be optional. Currently, even though many colleges ignore writing test scores, all students must take the writing portion of the test.
• Reading sections, like the writing section, will see a shift in focus so that students must cite evidence from passages to support their answers.
• The point scale will return to 1600, as it was before the writing test was added in 2005, when the scale changed to 2400. Those who take the writing test will receive a separate score for that.
• Points will no longer be deducted for incorrect answers on the multiple choice part of the test. Currently, one-quarter of a point is deducted for each incorrect answer, so students engage in strategy games about when they have eliminated enough incorrect answers to make it worthwhile to guess.
• Vocabulary words will eliminate "sometimes obscure" language that has been dominant and will be replaced by words "that are widely used" in college and the work place. In testing of words, the College Board will stress those for which meaning depends on context. The College Board gave as examples of such words "synthesis" and "empirical."
• Passages of writing used for various parts of the exam will be texts from significant moments in American history or science, not the somewhat random selections that now appear. Each exam will feature works such as the Declaration of Independence or a selection from the Federalist Papers, or Martin Luther King Jr.'s "Letter from a Birmingham Jail."
• Mathematics questions will be narrowed to focus on three areas: "problem solving and data analysis," algebra and "passport to advanced math." The College Board says that the much wider range of topics now featured will be eliminated so that students can study specific areas and feel confident they will be tested.
• Print and digital versions of the SAT will be offered; currently the test is paper only.

Reaction to the changes was mixed, with many cheering the demise of the current writing test and pledges to make the SAT more easily understood. But skeptics questioned whether the changes were going far enough -- and whether they represented an attempt to become more like the College Board's chief competitor for the SAT, the ACT.

Other changes announced Wednesday go beyond the test itself. For example, the College Board announced a collaboration with the Khan Academy in which the latter organization (which produces well-regarded educational videos) will produce 200 videos that cover topics related to the new SAT. (And videos will also be produced to help those students taking the current version of the SAT.)

As is the practice for the Khan Academy, those videos will be available free. Historically a criticism of the SAT has been that wealthier students can afford coaching. Sal Khan, founder of the Khan Academy, issued a statement in which he said: "For too long, there's been a well-known imbalance between students who could afford test-prep courses and those who couldn't." Khan said that his services would make "truly world-class test-prep materials freely available to all students."

The College Board also announced a plan to provide four fee waivers for college applications for all low-income students. Most colleges already waive fees for low-income students, but some experts have said that eligible students don't even apply for the waivers, and are discouraged from applying.

For the College Board, this overhaul of the SAT comes at a challenging time. An increasing number of colleges have gone "test optional" in admissions. Just last month, a report found that submitting or not submitting test scores at test-optional colleges makes "virtually no difference" for a college in making admissions decisions based on high school grades.

Further, the SAT has been losing market share to the ACT. Historically the SAT has been the dominant player, but for the last two years, more high school seniors have taken the ACT than have taken the SAT. The ACT, historically strongest in the Midwest and the South, has seen growth nationwide. But it's also the case that for many of the country's most elite colleges, the SAT remains the admissions score submitted by most applicants.

Advocates for the ACT have historically said that it is less coachable and more directly linked to students' performance in rigorous high school courses.

The new president of the College Board, David Coleman, who set off the process of revising the SAT, played a key role in the development of the Common Core State Standards, which ACT has said its test is well designed to reflect.

In his remarks Thursday, Coleman acknowledged many problems with the SAT. But he also said that all of the problems with the SAT in its present form are also problems for the ACT.

"It is time to admit that the SAT and ACT have become disconnected from the work of our high schools," Coleman said. "And we've also been listening to students and their families for whom these tests are often mysterious and filled with unproductive anxiety. They are skeptical that either the SAT or the ACT allows them to show their best work. And too many feel that the prevalence of test prep and expensive coaching reinforces privilege rather than merit."

Writing and Other Revisions

The changes in the writing section may be most immediately visible to test takers.

The writing section was added in 2005 after a revision of the test prompted by a speech in 2001 in which Richard Atkinson, then-president of the University of California, suggested that his system's campuses end SAT requirements for admissions. That prospect prompted the College Board to adopt a number of changes (and kept the University of California on board requiring the SAT).

But the format -- in which students simply respond to a prompt, without regard for facts -- has frustrated many writing experts. The National Council of Teachers of English issued an analysis criticizing the writing test. Les Perelman, a writing instructor at the Massachusetts Institute of Technology, boasted of how he coached students to write nonsense essays that still received good marks from SAT graders.

The essay approach announced today presumably would not be as vulnerable to students writing silly statements and getting high scores.

In its press materials, the College Board said that admissions officers were "split" on the value of the writing test, with some finding it "useful" and others not.

Not only is the College Board changing its approach to the essay, but it is starting an awards program, which it says will be modeled on the Pulitzer Prizes, for student analytical writing.

In an interview, Perelman  said it was good to drop the current form of the writing test. "Getting rid of what was a horrible writing test is in itself good," Perelman said. He said that it remains to be seen whether standardized testing is the best way to evaluate writing. Perelman said colleges would be better off asking applicants to submit all drafts (including teachers' comments) on an actual paper or papers they wrote in high school.

Jerome A. Lucido, executive director of the Center for Enrollment Research, Policy and Practice at the University of Southern California, said he was pleased to see the writing test change, but questioned why the College Board was dropping writing as a requirement. "If the writing test was flawed and a better one has been produced (which looks to be the case), why make it optional?" Lucido asked in an email. "Writing is very important, and I very much welcome an improved test."'

Of course, one reason the College Board may be reluctant to require the writing test is that many college admissions leaders -- including some who value the SAT -- have simply ignored it. Michael Sexton, vice president for enrollment management at Santa Clara University, said that his institution has never looked at scores on the existing essay on the SAT, "so we won't miss it." He said that, generally, the changes announced Wednesday made sense. And he said he values the SAT as helping admissions officers make decisions, especially in science fields.

A number of other admissions leaders agreed, although frequently with caveats about one or more changes.

Seth Allen, vice president and dean of admissions and financial aid at Pomona College, said via email that "there appears to be more of an emphasis on measuring critical thinking skills in this new version of the test. There also appears to be more emphasis on connecting the material to actual learning rather than memorization and test preparation. Both are steps in the right direction."

But Allen said he was "less keen on the idea of including American-centric passages in every exam. This creates a potential issue for non-U.S. applicants who may not have been afforded the opportunity to study U.S. history or the American context in their own national curriculum."

More Like the ACT?

Many of the changes announced Wednesday were portrayed as making the SAT more closely tied to the curriculum in rigorous high school courses, and less tied to test-taking skills. Privately, some admissions officials said that they saw the College Board trying to shift direction toward the ACT model -- even as Coleman described both tests as needing change.

Akil Bello, co-founder of Bell Curves, a test-prep company in New York City that works primarily with low-income students, tweeted during Coleman's talk that "Coleman keeps dragging the ACT under the bus with him each time he talks about the flaws in the #SAT," Bello wrote. Asked about this in an interview, Bello said that he doesn't see the ACT having the same problems as the SAT. "ACT feels more comfortable to most students in high school," he said. He said Coleman was "marketing" to imply that all tests have the same problems.

Other observers agree. Seppy Basili, vice president of Kaplan Test Prep, said of the College Board that it "was making a lot of changes that mirror the ACT."

Perhaps not surprisingly, the ACT concurs. Paul Weeks, vice president of customer engagement, said that "my honest take-away is that most of the changes kind of validate what we've been doing for a long time. I'm delighted they think assessments ought to reflect what students are taught."

Weeks said that he thought Coleman made "an unfair characterization" in suggesting that both the SAT and ACT needed more alignment with the curriculum. Weeks noted that, for many years, the ACT has conducted regular surveys of high school teachers to find out what they are teaching, and of college professors to find out what they expect of freshmen, and that the ACT is regularly tweaked to reflect shifting needs. He stressed that educators see the ACT as very connected to the curriculum, and thus less coachable.

Joseph Soares, professor of sociology at Wake Forest who has written extensively about standardized admissions tests and has advocating dropping them said he saw Wednesday's announcement as an attempt "to shore up the weakened position of the SAT relative to the ACT in the test-market, and to keep colleges from going test-optional."

He said the College Board essentially was conceding major failures in recent years -- but was hoping to regain market share. "The College Board has admitted that the revised 2005 SAT was a failure. It failed educationally and it was a failure for equal opportunity, but it succeeded in one respect. The 2005 SAT kept the University of California from dumping the test," Soares said. "The statement offered today by the president of the College Board has that same type of goal in mind."

A Shift on Test Prep

Many critics of standardized testing have historically pointed to the role of test-prep companies, saying that they create inherent advantages for the wealthier students, who can afford them. Historically, the College Board has downplayed the influence of test prep, saying that most such coaching results in only modest gains and shouldn't have a material impact on admissions. (A 2009 report by NACAC suggested otherwise.)

Coleman's remarks on Wednesday were quite critical of the test-prep industry. "The College Board cannot stand by while some test-prep providers intimidate parents at all levels of income into the belief that the only way they can secure their child's success is to pay for costly test preparation and coaching," he said. Coleman's solution was the arrangement announced with the Khan Academy.

Khan Academy materials are widely praised by educators, and the link with Khan won praise. But some noted limitations. Bello, of Bell Curves, said that while he supports the work of the Khan Academy, he works with many students who don't have Internet access or laptops at home, or have much access to them outside of school. "Khan may not help them," he said.

Chad Troutwine, co-founder of Veritas Prep, said he agreed with Coleman that test-prep services favor wealthy students, and he praised the College Board's leader for being forthright about the issue. Troutwine said he applauded the idea of the College Board working with Khan Academy to provide test prep, free, for everyone.

But Troutwine said that there is no way that Khan, however high the quality of its videos, will replicate the kinds of services he and others provide. His approach to the equity issue has been to get various groups to foot the bill so he can offer Veritas Prep to low-income students, and to support that work himself. With fees, "we let the wealthier parents subsidize the others," he said.

It's important to admit, he said, that "more privileged children are still going to get a higher value of test prep" from paid services than will other students.

Basili of Kaplan Test Prep said he had no worries about business because of Coleman's statements. Basili said that the last two years in which the College Board changed the SAT -- 1994 and 2005 -- were by far the years in which Kaplan Test Prep saw the greatest business growth.

Reaching Out to Critics

To testing skeptics, the changes announced Wednesday won't make a huge difference. A statement from the National Center for Fair & Open Testing (FairTest), which has long criticized the College Board, said that "the revised test is unlikely to be better than the current one. It will not predict college success more accurately, assess low-income students more fairly, or be less susceptible to high-priced commercial coaching courses."

But while FairTest isn't endorsing the changes, it was consulted -- and several admissions leaders noted that Coleman reached out much more than others, to rank and file admissions leaders, and to testing critics, to talk as the College Board put together the SAT revisions.

Robert Schaeffer, public education director of FairTest, said that Coleman set up an hourlong phone call with him, which was followed by email exchanges and shorter conversations. And he said he was invited to a conference call with College Board officials next week.

Perelman, the MIT professor who coached students on how to dupe the College Board into giving good ratings to bad essays, said he didn't expect outreach. But shortly after Coleman was named as College Board president, he invited Perelman to talk, and Perelman said he found Coleman engaged and receptive to his critiques.