Softening Claims of the Marshmallow Test

Stanford University’s famous “marshmallow test,” that adorable assessment of willpower that has fascinated educators and social scientists for decades, may not necessarily hold the key to prosperity, health and happiness, new research suggests.

Instead of simply bolstering young people’s abilities to resist temptation, the research finds, adults may need to consider deeper interventions -- such as helping mothers earn college degrees, improving parenting skills and home environments and, well, pulling children out of poverty.

In other words, success may be a bit of a heavier lift than simply training preschoolers not to eat a marshmallow off a plate for 20 minutes.

In a new study using longitudinal data from the National Institute of Child Health and Human Development (NICHD), researchers at New York University and the University of California, Irvine, found that interventions inspired by the marshmallow test, which successfully boost children’s early “delay ability,” might have no effect on later life outcomes. In other words, teaching children to delay gratification may not necessarily be the key to a happier, more productive life.

Much of the difference between children who succeed a decade or more later and those who don’t, they say, can be explained by more complex factors such as a family’s socioeconomic status, parenting skills, home environment and whether a child’s mother earned a college degree.

Using the federal data, NYU’s Tyler W. Watts, along with UC Irvine’s Greg J. Duncan and Haonan Quan, replicated the well-known findings by former Stanford University researcher Walter Mischel, who in the 1960s got the idea to gather children at Stanford’s Bing Nursery School and present them with a variety of sweet treats, including marshmallows.

Mischel and his colleagues told children that they would be left alone in a room with the treat or, in some cases, a picture of the treat. They were told they’d get permission to eat it immediately by ringing a bell -- or they could wait for up to 20 minutes until an adult returned. If a child waited, she got two treats instead of one.

Mischel and subsequent researchers later found that the longer children resisted the treat, the better they performed on several key measures, including academics. Eventually, the children who waited earned more and were healthier. Subsequent research found that those children also performed better than the bell ringers in avoiding problems like drug addiction and jail time.

The conclusion, it seemed, was clear: bolster children’s early self-control and the benefits will accrue.

But Watts, an assistant professor of research and a postdoctoral scholar at NYU's Steinhardt School of Culture, Education, and Human Development, said the earlier experiments had problems. For one thing, the sample of children was small and highly selective -- children were originally recruited from those with Stanford ties, and though Mischel’s original sample included more than 650 preschool-age children, follow-up studies focused on a much smaller group. Of 653 children from Mischel’s original study, researchers were able to find only 185 for follow-ups, Watts and his colleagues note.

“The statistical approach used to do that work could be updated,” Watts said in an interview. “People haven’t really looked at this with the type of scrutiny you’d expect today, if those findings were to come out as newly published material.”

Watts said he and Irvine’s Duncan have long been trying to figure out what skills schools can boost in early childhood “to get the most bang for your buck.” Helping children delay gratification has long seemed a likely strategy.

“If you can tune up a kid’s ability to delay gratification or to wait longer on something like the marshmallow test, that may unlock a lot of benefits later on in life,” he said.

They’d recently learned that the NICHD study, which has tracked about 1,300 young people from birth in 1991, included a version of Mischel’s marshmallow test “sitting there in the data. Someone had decided to collect it.” The only known reference to the NICHD marshmallow data, he said, comes from a 2013 study by University of Pennsylvania researcher Angela Duckworth, whose research on the topic has popularized the idea of developing "grit" in young learners, an idea that a few other researchers have questioned recently. Duckworth could not be immediately reached for comment.

In the NICHD study, researchers gave the children, then aged 4 and a half, a seven-minute marshmallow test -- much shorter than the original. Researchers measured children’s ability to wait down to the second.

Sifting through the data, Watts and his colleagues found that about 55 percent of children hit the seven-minute mark without eating the treat, but that kids from wealthier families actually found it harder to wait that long. By contrast, many more children whose mothers had a college degree made it to seven minutes -- 68 percent, compared with 45 percent for children whose mothers did not finish college.

As with Mischel, they found that waiting correlated to better outcomes later in life. Compared to children who couldn’t wait even 20 seconds before ringing the bell, those who waited up to two minutes scored higher in basic skills, both in first grade and in high school. For children who could wait the entire seven minutes, the difference grew more.

But here's the sticky part: when Watts and his colleagues factored in family characteristics, the differences began to fall away. In most cases, by high school, outcomes between the bell ringers and the children who waited for that second marshmallow were "rarely statistically significant."

Grouping together children whose mothers hadn’t completed college showed that all had similar achievement levels at age 15 -- the children who could wait just 20 seconds and those who could wait the entire seven minutes.

Similarly, there were few achievement differences among children whose mothers had completed college, no matter if they rang the bell or waited the full seven minutes.

The results, Watts and his colleagues write, create further questions about what the marshmallow test actually measures. Could focusing simply on interventions that improve a child’s ability to delay gratification be doomed to failure if teachers don't address “more general cognitive and behavioral capacities”?

In an email, Mischel noted that the data set used by Watts et al. found "a significant positive correlation between delay time and academic achievement" that strengthens earlier findings.

He also said years of research by him and his colleagues, as well as by others, have found that "a child's ability to wait in the 'marshmallow test' situation reflects that child’s ability to engage various cognitive and emotion-regulation strategies and skills that make the waiting situation less frustrating. Therefore, it is expected and predictable, as the Watts paper shows, that once these cognitive and emotion-regulation skills, which are the skills that are essential for waiting, are statistically 'controlled out,' the correlation is indeed diminished."

He added, "The 'marshmallow test' was developed to find a method -- a window -- that allows us to see how people, especially children, manage to deal with the frustration of waiting for something they really want to have. This window opened the way for many experiments that identified the conditions and mental-emotional strategies and skills that make this challenge manageable or not. The long list includes, for example, trust in the promise-maker, and diverse cognitive and 'cooling' strategies to make the waiting easier."

Indeed, subsequent research has used Mischel’s paradigm to test, among other indicators, whether children in unpredictable circumstances might actually be acting rationally by eating that first marshmallow without waiting.

In a 2013 University of Rochester study, for instance, researchers offered a group of children the chance to draw a picture with a small set of crayons or to wait, marshmallow style, for a nicer set of art supplies. But there was a twist: in a few cases, the children who waited didn’t get the nicer supplies -- instead they got an apology from adults and the same old lousy set of crayons.

Presented moments later with an actual marshmallow and Mischel’s famous offer, children who’d been tricked into waiting for the phantom art supplies were less likely to wait for that second marshmallow.

Researchers concluded that these children were acting rationally -- they’d just had their beliefs about the “stability” of the world tested, and they likely ate that first marshmallow because they correctly reasoned that waiting, i.e., trusting that adults would do what they said they’d do, wouldn't pay off.

But Mischel said his research and subsequent work based on it "does not suggest that the method is a crystal ball that predicts our future, or that training children to wait for marshmallows is a panacea. A close reading of the Watts et al. paper adds to this understanding. Unfortunately, our 1990 paper’s own cautions to resist sweeping over-generalizations, and the volume of research exploring the conditions and skills underlying the ability to wait, have been put aside for more exciting but very misleading headline stories over many years."

Whatever one concludes from Mischel's findings, Watts said, the test has been “extremely influential” to developmental psychologists. “It’s told us a lot about how young children approach really interesting problems and what’s going on in kids’ minds at a really young age.”

He said researchers and educators need to be more careful when making pronouncements about ideas like self-control, and to temper our expectations. “If we think that this is actually an important skill that we try to target for kids to unlock later life outcomes,” he said, “our results suggest, ‘Probably not.’”

The larger problem, he said, boils down to this: psychologists need more research that tackles not just correlation, like the marshmallow test and its suggestion of future self-discipline, but causality.

“What we’re really trying to scratch at is a causal effect. And the only way to really identify what kinds of changes in kids’ skills and competencies are going to have causal effects on their later development is through experimental work. That’s the only way to really get at that.”

The study was published May 25 in Psychological Science.