Grading the AP Curriculum

The College Board announced the results of its audit of Advanced Placement courses Monday, saying that most AP courses meet college-level standards, and that the review process had helped thousands of others reach appropriate standards. At the same time, the process identified thousands of courses that didn't earn the right to keep identifying themselves as part of the AP program.

While board officials hailed the results of the audit, some critics of the AP courses were more reserved. Generally, the board was praised for undertaking the effort and for taking steps to assure the academic rigor of the AP courses. Even some AP skeptics said that the process will probably improve the courses. But some of those skeptics said that the College Board's process did not in fact answer the key question about the courses: Are they college level?

The plan to conduct the audit was announced in 2005, at a time that the AP program was experiencing growing popularity among high school students and growing skepticism among some colleges. Students, who view AP courses as key to winning admission to elite colleges, are enrolling in the courses in greater numbers -- and urging their high schools to offer more of them. Colleges have generally been appreciative of anything that makes the senior year of high school more rigorous, and generally applaud the AP program for engaging and challenging students. But many professors were expressing doubts that AP was truly college level. And others were questioning the fairness of using AP in admissions when low-income high schools typically offer far fewer AP courses.

In the audit, teachers of AP courses were told that for their offerings to continue to use the AP name, they had to submit a syllabus showing that key standards were being followed. Those standards cover topic areas, textbooks, lab time for science courses, and other factors.

In total, 134,000 syllabuses for AP courses were submitted by teachers, and 67 percent were approved on the first review (involving college professors). Of the 33 percent that were not immediately approved, some have since been approved or should soon be approved because only minor changes were needed -- an additional textbook, or some additional subject matter. College Board officials said that they didn't know how many of these courses are likely not to be approved in the end, and stressed that their goal was to get the courses improved to the level that they could be approved.

Beyond the courses in the 33 percent that don't raise their standards, others are also losing the AP label. At 2,081 high schools that had been offering AP, officials decided not to submit to the audit after they reviewed the standards that would be applied. These high schools offered an average of 5 AP courses each and will now not offer any. In these cases, the high schools realized that they were sufficiently far off that it wasn't worth going through with the audit. College Board officials said that they believe other high schools that had been using the AP name in ways that didn't correspond to any AP program (AP gym or AP study hall, for example) have stopped using the designation, although there are no data about that.

Trevor Packer, vice president of the AP program, said that the "key" findings were about those high school teachers who did submit their courses for review. In a number of cases, they also reported on changes they made -- after receiving the standards and before submitting their courses -- that resulted in more rigorous AP courses.

According to the College Board, 17,000 teachers reported using the audit process to prevent reductions in lab or instructional time that would have affected their courses, and another 16,000 were able to obtain new college-level textbooks. Packer said that the textbook issue was crucial because, in fields where material and college expectations change, he was concerned that some AP programs were not keeping up.

The data also showed clear differences in the quality of AP offerings at high schools that are high and low on the socioeconomic ladder. Only 3 percent of the wealthiest high schools had to improve textbooks as part of the process, while 22 percent of low-income high schools were found to be using deficient texts. The rate at which laboratory time had to be added was twice that at poor high schools than at rich ones.

The subject area with the highest approval rate was calculus, which Packer attributed to the wide consensus about what calculus should cover. The area with the lowest approval rate was Japanese, which Packer noted was new to the AP program. Economics and government and politics courses also had relatively low rates of approval.

Going forward, Packer said that courses will need to be audited again whenever teachers change or when substantial changes are made in the AP curriculum. He said this will avoid a problem that the program experienced last year, when it alerted teachers of the comparative government AP courses that the test would have material on Iran. That was added out of the view of college professors advising the College Board that such courses should include some comparative study of Islamic governments. Many teachers apparently didn't change their courses, and their students had no relevant knowledge when they hit the Iran portion of the AP test, Packer said.

The test of the success of the audit, Packer said, will be when scores are released for those students taking the post-audit AP courses now. He said he hopes and expects that there will be more high scores, suggesting more students are receiving a better education.

As the audits have gone on, reports have surfaced of high schools in which identical courses aren't evaluated in the same way. The Washington Post reported in September on high schools where teachers were frustrated by the audits. At one high school, three teachers submitted the same syllabus -- one teacher's course was accepted, one was rejected with three suggested revisions, and one was rejected with eight suggested revisions.

Packer said that overall teacher reaction has been positive. While he said he couldn't speak to individual cases, he said that the College Board did "inter-reliability studies" in which the same syllabus was given to multiple professors for review, and that in 97 percent of cases, the professors made the same decision.

Robert H. Tai, an associate professor of education at the University of Virginia, is among the scholars who have questioned the central claim of the AP program that it certifies college-level work. He was co-author of a study last year that found that in science courses at 63 four-year colleges and universities, students who had taken AP courses -- including students who did well on them -- did only marginally better than students who hadn't taken AP. Other factors, such as the rigor of mathematics in high school, were found to have a much more significant impact on college performance.

Tai said Monday that there is a lot he likes about the College Board audit. He said that the process seems to have spurred thousands of teachers to improve their courses, and that since he believes AP courses can add substance to the senior year of high school, that's great, Tai said. He also applauded the College Board for focusing on improvements rather than taking a "punitive approach."

But Tai said that nothing in the audit process demonstrated that the courses are college level, nor would an increase in AP scores this year, if that happens. "It's all too insular," he said of the process. More studies like the one he released last year are needed, he said. The only way to tell if AP courses really are providing a college-level experience is to see if students who place out of a college course because of AP perform as well as those who take the material in college. The evidence isn't compelling there. For instance, a study at the Massachusetts Institute of Technology found that students who earn top AP scores and place out of introductory courses at MIT ending up having “difficulty” when taking the next course. That led a faculty panel to propose restricting the use of AP to place out of courses to calculus, the one field where the negative impact was not found.

Tai said he doubts that students who take an AP course and earn a 5 are really at the same level as those who earn an A in college. "The College Board wants to suggest that this is as good as or can be taking in lieu of college courses, but there's no evidence," he said. "I think having students go through these courses is a good thing, but it's troubling to have these broad claims made without evidence. There's just no accountability."