Search News


Browse Archives

News

SAT Controversy Continues

July 21, 2006

Share This Story

FREE Daily News Alerts

Advertisement

For months now, the College Board has been saying that it has new measures in place to prevent a repeat of the embarrassing scoring errors on the October SAT. On Thursday, the board released a much-awaited report, which said -- yep -- that the board has new measures in place to prevent a repeat of the embarrassing scoring errors on the October SAT.

The board released the report at the insistence of a powerful New York State senator, who had threatened legal action to obtain it, and to whom board officials promised the report months ago. While some critics of the board said it was good that the report was finally out, the emphasis of the study -- on steps taken and that potentially could be taken to improve scoring accuracy -- bothered many. They said that the report revealed little about how the scoring errors took place or why they took so long to discover and report.

The report was prepared by Booz Allen Hamilton, a consulting firm that found that "the current process [of scoring] is reliable and has prudent controls in place to safeguard scoring accuracy for those marks made in accordance with test directions." The report also found that "operational changes made by College Board in response to the October administration further improved process reliability by introducing scanning redundancy, more frequent scoring checks, an environmental acclimation period to eliminate the effects of humidity, and other safeguards."

While the report largely praised the College Board's system, as improved, it also outlined 16 "secondary risks" that -- while not posing grave dangers to reliability -- could cause problems. These risks include items related to the actual scanning process ("bubble alignment error due to paper manufacturing") and to the "upstream" process when exams are moved and scored (the risk of packages being tampered with prior to shipping). For each of these risks, the report outlined steps that the College Board could take to mitigate them.

In a statement, Gaston Caperton, president of the College Board, praised the report and pledged to deal with all of the risks that have been identified. "The College Board is addressing every one of those risks," Caperton said.

To the extent that the report didn't satisfy critics of the College Board, it wasn't so much what was in the report but what wasn't. The Booz Allen Hamilton study was forward looking in that it emphasized risks ahead and how to deal with them. It didn't contain the kind of analysis many SAT skeptics have been seeking on why thousands of scores were incorrect and how the College Board handled the incident.

Sen. Kenneth P. LaValle, chairman of the New York Senate Higher Education Committee, criticized the College Board Thursday for holding on to the report for so long without releasing it (the report is dated May 26) and for not figuring out why the mistakes happened in the first place.

"The board's lack of accountability in this matter goes to the heart of the arrogance that exists when an organization basically holds a monopoly in a certain market," LaValle said in a statement. "This report still does not answer why such a large number of testing errors occurred on the October 2005 SAT. In addition, the board has not provided an explanation of why it took several months to inform over 4,000 students of inaccurate test scores."

LaValle renewed his call for "strong accountability" for the College Board. Unlike many critics of the College Board, LaValle may have the clout to force changes. He is the legislative father of the "truth in testing" movement in New York State, which led to earlier reforms of the testing industry -- largely against its will.

The National Center for Fair and Open Testing responded to the report by releasing a list of questions that the study did not answer. Among them: What was the root cause of the problem? When did the College Board receive the first request for hand scoring of the October SAT? How many weeks did it take for the board to respond to that request? When did the College Board first discover a broad problem? Why was there a delay between that time and when test takers and others were informed of the problem?

FairTest and Senator LaValle have a history of sparring with the College Board. But the continuing controversy over the SAT also has many admissions deans frustrated. Jeff Rickey, dean of admissions and financial aid at Earlham College, said he couldn't understand why the College Board hadn't released the study earlier, and speculated that it might be because of "their habit of displaying an abundance of arrogance and a shortage of humility."

Rickey offered a suggestion: The College Board should change the name of its yearly meeting of members in November from "Annual Forum" to "Asking Forgiveness."

See all postings »
Advertisement
Advertisement

Matching Jobs

Comments on SAT Controversy Continues

  • Posted by math prof on July 21, 2006 at 11:45am EDT
  • The College Board commissions a report on itself. The report praises the College Board. Why am I not surprised?

  • Posted by Mick on July 21, 2006 at 8:35pm EDT
  • A more basic issue concerning the SAT (and the ACT)is their basic utility in making admissions decisions at all. While both testing agencies make claims of validity for the tests, the questions of gender, race, and cultural bias have not gone away. Neither is a reporter or a predictor of work habits, which probably have more to do with college success than the basic reading and math skills tested. Both maintain that admissions decisions should be based solely upon such test results, but one has to wonder what role such test results should play at all.

  • Posted by Patrick Mattimore , Teacher on July 21, 2006 at 8:35pm EDT
  • Interesting that in numerous media accounts one of the persistent critics of the College Board, Robert Schaeffer of Fair Test, calls this 75-page report a red herring. Fair Test would love to see the end of the SAT altogether. Kenneth Lavalle, the NY Senator, says he wishes the report got at the causes of the scoring mistake instead of being a risk management study. Both of these guys carry some pretty heavy prejudgment baggage into these critiques.
    The real red herring is that the scoring gaffe is being used to attack the SAT test itself. The College Board makes itself an attractive target by being generally arrogant. But until someone comes up with a better standardized predictive measure, we're better off to leave the messenger alive and insist on working with the CB to improve the test.

  • Posted by Patrick Mattimore , Teacher on July 22, 2006 at 2:45pm EDT
  • "Both maintain that admissions decisions should be based solely upon such test results, but one has to wonder what role such test results should play at all." (from Mick).
    This is the kind of misinformation that is widespread. Neither the makers of the SAT nor the ACT maintain that the tests should be the sole criteria in admissions. In fact, the College Board (who bring us the SAT) state on their web site that "test scores supplement, not replace, other important information." CB makes a variety of other cautionary statements re. use of the SAT (www.collegeboard.com/research) but, of course, the weight colleges choose to give the SAT's are up to them.
    What the CB does maintain is that the "SAT Reasoning Test scores, combined with high school grades, result in more accurate predictions of college success than either measure alone or any combination of other measures."

  • predictor of what?
  • Posted by Jesse Bacon , Teacher at Chicago Public Schools on July 30, 2006 at 4:45am EDT
  • I think that even if SAT's combined with grades are an accurate predictor of college "achievement", that says more about what are colleges are rewarding than the virtue of the tests themselves. If our colleges are churning out gradegrubbing, uncritical, social climbers, than it stands to reason test scores would be a good predictor of that. if nothing else, it tells us that colleges are teaching only people who already learn in the dominant manner. However, if we want to know if our colleges are helping students to think for themselves, become ethical member of society, and pursue their intellectual passions, tests would be absolutely worthless as a predictor. We need to question our assumptions of what constitues "achievement" and what kinds of assessment are useful to meeting those goals.

  • The College Board's OTHER debacle
  • Posted by Truth Seeker on August 9, 2006 at 1:45pm EDT
  • Those wondering about how the College Board scores the SAT, should look into how they judged the 2004 Siemens-Westinghouse Science Competition. Was this a great example of competance or what?