You have /5 articles left.
Sign up for a free account or log in.
The Princeton Review, a leading test-prep company, has agreed to stop using claims about average score gains in its marketing materials. While company officials say that they believe the claims were accurate, and that they were preparing to move away from such claims without outside prodding, the decision came after an investigation by the National Advertising Division of the Council of Better Business Bureaus, which found the decision to stop making such claims to "be necessary and appropriate." (The organization acts as an arbitrator among companies that agree to have complaints probed.)
The inquiry was based on a complaint from Kaplan Inc., a major competitor in the test-prep industry. Kaplan asserted that Princeton Review had no basis to talk about score gains because the start point for measuring gains was generally determined by diagnostic tests, while the end point was a live test. Critics have said that they believe test-prep companies' initial tests yield low results, encouraging people to sign up for courses and to credit the companies for large gains later.
Whatever the accuracy of the claims, they have been quite visible. Examples cited in the investigation of the Princeton Review include: "In fact, our students improve their GMAT scores by an average of 90 points" or "Our students improve their GRE scores an average of 206 points" or "Our SAT Ultimate Classroom students average a score improvement of 255 points."
Scott Kirkpatrick, president of Princeton Review's Test Prep Division, said that Kaplan's complaint may have helped contribute to a sense that "the timing was right" to stop making score gain claims. But he said that the shift also reflects internal thinking about "who we want to be as a company." He said that the company wanted to be "a true education company. Instead of talking about beating specific tests, we are about preparing all students for their next step on their own terms."
He said that every student "learns in a different way" and that learning "is not all about score improvement." He said some students need and want large gains, and others need and want small gains. While test-prep companies should assure prospective students and their families about the results of their services, they should talk broadly about education, Kirkpatrick said.
For Princeton Review, which has prided itself on being the bad boy of test prep, that talk represents quite a shift. Kirkpatrick said that the company's original focus was on "serving the upper echelon of students," while today, the company wants to reach everyone. And it may also be harder for Princeton Review to focus on helping people "beat" the tests when the company is (like Kaplan before it) now in the business of providing course offerings and degrees, not just test-prep services and college guides.
Kaplan praised the findings by the advertising group and the decision by Princeton Review. While Kaplan has not made comparable claims, it too has adjusted marketing in recent years by ending the use of testimonials in which test takers talk about their large gains after using Kaplan services. "We are moving away from even these individual testimonials in an effort to further move the discussion away from specific scores and in the direction of quality of programs and the student experience," a spokeswoman said.
Many admissions officers and testing critics have worried for years that the test preparation industry favors wealthier students, and also that test-prep companies may exaggerate their impact, leaving many applicants feeling that they have no choice but to sign up. Robert Schaeffer, public education director of the National Center for Fair and Open Testing, said the evidence has long suggested that test-prep companies overstate the impact of their services. He said that the announcement from Princeton Review is positive, but that "everyone else in the coaching business remains free to fabricate score gain claims and promote them to the hilt."
Last year, the National Association for College Admission Counseling released an analysis on the impact of test-preparation services that backed the claims of companies that they do produce gains on the SAT. But the research suggested that the gains are relatively small -- gains that theoretically shouldn't matter much in admissions decisions. But NACAC also found evidence that at plenty of colleges, these kinds of gains could make a difference.
David Hawkins, director of public policy and research at NACAC, called the agreement by Princeton Review to stop the use of score gains in marketing "an important" action and "a step in the right direction." He said that NACAC commissions that have studied standardized testing and test prep have heard from people who believed that there are "false diagnostic tests that led to inflated test score improvements" but that NACAC was never able to document the issue. He said that the inquiry by the advertising group "offers further evidence that students need better information, at a minimum, to protect against misrepresentation with regard to test preparation."
While Princeton Review and Kaplan may not be making such claims, others are. Many of the boutique companies that have proliferated in the test-prep industry in recent years have Web sites full of testimonials about large gains in scores and promises of minimum gains.
The testimonial page of Knewton, for example, leads off with Alex R. saying "Thanks for the 240 point increase." And the comparison it offers on the GMAT test-prep course plays up a money-back guarantee for anyone who doesn't see a 50-point jump in scores.
Josh Anish, a senior editor at Knewton, said that the company measures gains in several ways. If a student has taken a test prior to enrolling in one of Knewton's programs, that test is the base. If not, the company does have diagnostic tests. He said that because the company hires many people who used to work for the testing companies such as the Educational Testing Service, Knewton is confident of the accuracy of its diagnostic tests.
Anish said that there is nothing wrong with boasting about score gains. He said that's what customers want. "We are proud of our courses," he said, and many of the clients are "a type-A audience and they want to know about return on investment."
