You have /5 articles left.
Sign up for a free account or log in.
Istockphoto.com/monkeybusinessimages
Phil Oreopoulos and his fellow researchers are confident that colleges should be able to use text messages and other forms of online communications to "nudge" students to study more so they perform better academically.
They just haven't been able to prove it yet through various studies, and their efforts offer a prime example of how new forms of educational research can be used to try to measure the impact of educational technology -- be it significant or nonexistent.
Their latest study, distributed last month by the National Bureau of Economic Research, admirably acknowledges failure -- or rather, in researchspeak, "null effects." “Lack of Study Time Is the Problem, but What Is the Solution? Unsuccessful Attempts to Help Traditional and Online College Students” notes right in its title that the scholars' attempts to drive thousands of in-person and online students to nearly double the number of hours they studied per week did not work.
Oreopoulos, a professor of economics at the University of Toronto, and a team of colleagues have for several years sought to apply the principles of behavioral economics to education, joining a cadre of scholars who are trying to measure whether interventions of various kinds can "nudge" students into altering their behavior to increase college going (applying for federal financial aid, applying to college, etc.).
More recently, he said in an interview, they've turned their attention to "whether there would be ways to incorporate these tools" once students get into college.
They persuaded a group of first-year economics instructors at the University of Toronto to require students to go online for warmup exercises at the start of a course, in return for a small grade enhancement. Using technologies the team built as part of their Student Achievement Lab, thousands of students at two of the universities’ campuses have over the last four years participated in a range of interventions: among them, working through an online module with information about successful study habits, preparing a required weekly study plan online and receiving periodic reminders via text message designed to keep them on track with their study plan. (These are the sorts of tools and approaches that numerous technology companies are using with their college customers.)
In general, the interventions have not meaningfully improved students' academic performance.
Along the way, Oreopoulos and the other researchers -- Uros Petronijevic of York University, Richard W. Patterson of the U.S. Military Academy and Nolan G. Pope of the University of Maryland at College Park, economists all -- collected reams of data about students' study times and academic performance. Among other things, they have found that the average student spent about 15 hours a week studying outside class.
"We see a lot of students are just not studying in the amounts we would expect, so maybe we shouldn’t be surprised why they don’t do so well," Oreopoulos said.
In the most recent iteration of their study, the researchers focused their efforts purely on trying to increase the number of hours students spent studying, to 25 from 15. "Our theory is, if we could move students to studying at a minimum around those hours, all other problems would be solvable in terms of their academics," he said.
Students were asked to do a range of things: read information about the links between study time and academic performance (including student testimonials), and then write about their own goals; create a weekly study schedule in which they would hit the goal of 25 to 30 hours a week (while building in time to be spent with family and friends); and enroll in a virtual coaching program. Study participants also received weekly text messages personalized to their goals and situations.
Many aspects of the intervention seemed successful. The vast majority of students did what was necessary to set the calendars up on their phones, for instance, and significant numbers of them (as many as two-thirds in some treatments) responded to the weekly messages. "A lot of feedback suggested that the program generated new interactions and support that would otherwise not have been going on," Oreopoulos said.
Which made it all the more disappointing when the data came back showing that the behavior of the participating students differed little from the control group that did not receive the various interventions. Students at Toronto increased their study time by about two hours a week on average, which the researchers deemed not to be meaningfully different, and students at the entirely online Western Governors University -- where a similar study was going on -- did not change their study time at all, as measured by log-ins and click data.
The researchers acknowledge that even the small gains they saw may have been exaggerated, since students' study time was self-reported. "We don't have a Fitbit for studying time yet," Oreopoulos said.
Not surprisingly, since the interventions produced little in the way of behavioral change, the researchers also found no meaningful improvement in study participants' academic outcomes.
"It may be that students aren't quite sure how to effectively spend the time they do study, and if they don't know what to do, that could lead to a bunch of conflicts and paralysis," said Petronijevic.
The researchers plan to tweak the study this year not just to promote more studying time, but to "really help resolve uncertainty around what they can be doing efficiently," Petronijevic said. "We're going to reverse engineer personalized suggestions for them, through text and email, to be clear about what they can be doing around time itself."
Despite the disappointing results so far, Oreopoulos and his colleagues are hopeful that technologies like the ones they've built can be used over time to improve student outcomes.
The intervention programs can be credibly tested and replicated (the scholars will make the tools they've developed through the Student Achievement Lab available to other colleges), and the technologies are improving all the time.
"We are optimistic that these types of online interventions, over time, will make a difference," Oreopoulos said. "And our ability to learn the truth about their potential impact is improving all the time, too."
All the technology in the world won't change student behavior, though, unless students can be persuaded of the benefits of studying and getting good grades.
"There's a lot of ambiguity" around "what do I gain from staying home on a Friday night? From getting a B average instead of a C?" he said. "I do know the cost" of staying home, missing out on a night of fun or time with friends. "It becomes a challenging problem to get one's head wrapped around the longer-term benefits."
"In that ambiguity, it's easier for a lot of students to settle, to convince themselves it’s not as important. That's the problem we need to solve."
