Multiple Measures, Better Outcomes

A recent study shows that using multiple measures to decide whether students belong in remedial coursework, as opposed to standard placement exams, results in more students taking and succeeding in college-level English and math courses.

The findings are outlined in a new brief and working paper released by the Center for the Analysis of Postsecondary Readiness, a research center focused on developmental education reforms. The center was created by the Community College Research Center at Columbia University’s Teachers College and MDRC, an education and social policy research organization.

The study, a follow-up to earlier research, tracked nearly 13,000 students who enrolled at seven community colleges in the State University of New York system in the fall 2016, spring 2017 and fall 2017 semesters. Some of the students were placed in English and math courses based on the colleges’ existing placement system, while others underwent an algorithmic multiple-measure assessment, a formula that considers a broader range of factors, in addition to or instead of placement tests, such as high school GPAs and the time since high school graduation. The goal of using multiple measures, a reform increasingly adopted by colleges across the country over the last decade, is to give students a better chance of being placed in courses that earn them college credit.

The study tracked students’ academic outcomes for nine terms after their placement tests, through spring 2021, including their enrollment and completion rates in college-level English and math courses and attainment of college-level credit. The data were disaggregated by race and ethnicity, gender, and whether students received Pell Grants, federal financial aid for low-income students. Researchers also took an extra-close look at students who would have had a different placement had they not been assessed using multiple measures.

Elizabeth Kopko, a senior research associate at the CCRC and an author of the working paper, said the study findings suggest that multiple-measure assessment strategies aren’t necessarily more accurate than placement tests at predicting which students are ready for college-level courses, but they improve the likelihood that students will succeed in these courses simply by ensuring more students have access to them.

“Just increasing access alone is the most important thing to help propel or improve student success,” she said.

Students assessed using multiple measures were placed and enrolled in college-level English and math courses at markedly higher rates than their peers who went through the usual placement process. These students were 18 percent more likely to be placed in college-level math and 73 percent more likely to be placed in college-level English. These students also had higher completion rates, earning at least a passing grade, in English courses across all nine terms, according to the working paper. In college-level math courses, these students initially had higher rates of completion, but completion gains weren’t statistically significant after the first term.

Meanwhile, students who otherwise would have been in developmental courses but were “bumped up” to college-level courses by the multiple-measures assessment had notably better outcomes than their peers, while those who otherwise would have been in college-level courses but were “bumped down” by the assessment fared worse.

Students put in college-level courses because of multiple measures were about nine percentage points more likely than similar students placed using the standard process to complete college-level math or English courses by the ninth term. Students bumped up to college-level English specifically were two percentage points more likely than their peers to earn a credential or transfer to a four-year university in that time period. In contrast, students bumped down to developmental courses were five to six percentage points less likely than their peers to complete college-level math or English.

Kopko noted that if multiple-measure assessments determined preparedness for college-level courses with more accuracy than placement tests, students bumped down to developmental education by the new method would complete college-level math and English at higher rates over the long run, bolstered by the extra academic preparation these courses are intended to provide.

But “the extra support offered through developmental education courses is not helping these students do any better than their peers who don’t get that extra support and are placed directly into college-level [courses]. And in fact, it’s quite the opposite … They’re actually doing worse,” she said.

Alexandra Logue, a research professor at the Center for Advanced Study in Education at the Graduate Center of the City University of New York system, said the stakes are high for students placed in remedial coursework when they could succeed in credit-bearing courses. Extra courses for no credit cost them extra time and tuition dollars and lessen their academic momentum.

“You’re making the degree more distant in time, and that is demotivating,” she said. For low-income and first-generation students or students with childcare responsibilities, “the longer it takes for you to get from starting college to finishing college, the more likely it is that things are going to get in the way. Things are going to happen in your life that make it very difficult or impossible for you to finish.”

Logue said she wished the study also examined outcomes for students in corequisite education, extra academic support intended to help underprepared students succeed in college-level courses, an offering that’s grown more popular at community colleges since the study started in 2016. But she praised the research for furthering knowledge on multiple-measures assessments and their advantages for students.

But multiple-measures assessments also weren’t a silver bullet, according to the working paper. The study found that the more varied placement process improved outcomes across student subgroups, but it didn’t close equity gaps between them.

“We generally find that there’s no change in the outcome gaps that pre-existed,” Kopko said. That might be “discouraging” at first glance, but “it’s important to take a step back and realize MMA was never intended to do anything of the sort. It really was a reform that was meant to impact the full student body,” so alone, it’s not enough. “We need to take into consideration the needs of certain populations, and we can’t just cross our fingers and hope that a universal reform is going to do what we want it to do with regard to equity.”

Hollie Daniels, a research associate at the CCRC and one of the paper’s authors, said for that reason, these assessments should be designed with equity in mind and paired with other kinds of supports that aren’t a “one-time intervention,” such as intrusive advising, tutoring or corequisite supports. She added that those designing these assessments should also be mindful not to set placement test score cutoffs or GPA requirements too high to give as many students the opportunity to take college-level coursework as possible.

The brief offered a series of recommendations for community college leaders. It suggested colleges place students in college-level courses whenever possible so more of them have access to these courses earlier and are more likely to complete and that they start by implementing “easy-to-adopt” multiple-measures assessment processes that don’t risk unnecessarily lowering students’ placements, among other suggestions.

Multiple-measures assessment doesn’t necessarily “decrease long-standing inequities, but it can be just one piece of a larger tool kit that helps to address some inequities that we see in higher education,” Daniels said.