The Decline of ‘Disruptive’ Science

Isaac Newton wrote to fellow scientist Robert Hooke in a 1675, saying, “If I have seen further it is by standing on the shoulders of Giants.” Centuries later, it remains generally understood that innovation builds on past science. So in this era of unprecedented research volume, breakthroughs should be increasingly common, right? Wrong, according to a new study finding that “disruptive” science is on the decline.

This trend downward “represents a substantive shift in science and technology, one that reinforces concerns about slowing innovative activity,” says the analysis, published this month in Nature.

Finding that the decline likely isn’t driven by changes in the quality of published science, citation practices or field-specific factors, the authors attribute this shift “in part to scientists’ and inventors’ reliance on a narrower set of existing knowledge.”

Relying on “narrower slices of knowledge benefits individual careers,” the paper warns, “but not scientific progress more generally.”

(Not) Forging New Directions

For their study, the authors analyzed 25 million papers in Web of Science databases published from 1945 to 2010, along with 3.9 million U.S. Patent and Trademark Office patents from 1976 to 2010. The idea was to analyze citations and texts to understand whether research and patents “forge new directions” over time and across fields. The authors replicated their findings using four more data sets: JSTOR, the American Physical Society corpus, Microsoft Academic Graph and PubMed, for an additional 20 million papers.

To measure disruption, the authors used a previously tested metric called the CD index, which characterizes how papers and patents change networks of citations in science and technology. The index, which ranges from -1 (consolidating research) to 1 (disruptive), is based on the idea that if a paper or patent is disruptive, the subsequent work that cites it is less likely to also cite its predecessors. For example, the paper says, James Watson’s and Francis Crick’s groundbreaking finding on DNA and Walter Kohn’s and Lu Jeu Sham’s major paper on quantum chemistry both received over a hundred citations within five years of being published. Yet the Kohn-Sham paper has an index of -0.22, indicating consolidation of previous work, the Nature paper argues. Watson and Crick have an index of 0.62, indicating disruption in science to that point.

Papers in life sciences and biomedicine, for instance, had an index of about 0.5 in 1950, compared to under 0.1 in 2010. Papers in the physical sciences, social sciences and technology saw similar declines. The same was true for chemical, computer and communications, drugs and medical, electrical and electronic, and mechanical patents.

Ultimately, the Nature study says, “We find that papers and patents are increasingly less likely to break with the past in ways that push science and technology in new directions. This pattern holds universally across fields and is robust across multiple different citation- and text-based metrics.”

Co-author Russell J. Funk, associate professor of strategic management and entrepreneurship at the University of Minnesota, Twin Cities, told Inside Higher Ed that disruptive science is “work that introduces breaks with the status quo and carves out new directions for scientific inquiry.”

Of the new paper, he said, “We find a lot of evidence that scientists are building off increasingly narrow bits of prior scientific knowledge. Over time, there’s a steady increase in the rate at which “everyone is citing the same stuff.”

This narrowing, he continued, “is associated with a decline in disruptive work.”

Is that a bad thing? Funk said that “a healthy ecosystem of scientific discovery probably needs a mixture of both work that is disruptive and work that builds on and improves the status quo of scientific knowledge. The question, then, is what is the right mix?” A “dramatic decline” in the disruptiveness of papers and patents, as demonstrated in the analysis, suggests that “the balance may be off.”

Implications and Explanations

What can concerned institutions and funding agencies do to encourage more innovation and disruption, if anything? Funk said that factors to consider include “funding higher-risk research that proposes and develops unconventional theories and methods.” Putting greater emphasis on the “quality of work rather than the quantity of papers” produced may also help, he added.

Funk’s analysis isn’t the first to explore whether innovative activity is really increasing. One 2020 study of 1.8 billion citations among 90 million papers across 241 subjects found that a “deluge of papers” does not lead to turnover of central ideas in a field, but instead to “ossification of canon.” That study, published in Proceedings of the National Academy of Sciences, also noted that “fundamental progress may be stymied if quantitative growth of scientific endeavors—in number of scientists, institutes, and papers—is not balanced by structures fostering disruptive scholarship and focusing attention on novel ideas.”

Concern—and various theories—accompany such findings. Among them: that the low-hanging fruits of scientific discovery have largely been plucked already, and that scientists nowadays need so much training to reach the forefronts of their fields that they lose precious research time getting there.

But Funk’s paper has attracted particular attention from scientists concerned that science today is more about pushing papers and chasing funding than it is about doing the kind of deep work that leads to breakthroughs.

Itai Yanai, a professor of biochemistry and molecular pharmacology at New York University’s Grossman School of Medicine who tweeted about the new study—arguing, in part, that science now encourages principal investigators to act as managers, leaving less time for innovation and disruption—told Inside Higher Ed, “The longer I have been a professor, the less time for science, unfortunately.”

Yanai said that “what really counts as science for me is working with the members of my lab and collaborators and also working alone,” but that this makes up about half of his working time. As for the rest of the time, he said, “There are so many obligations for a professor, including attending student committees, attending seminars, preparing and delivering lectures, writing and submitting grants, writing progress reports, traveling to present the work, reviewing manuscripts, reviewing grants, answering emails.”

A ‘Fundamentally Creative Process’

Asked about the implications of Funk’s finding and others like it, Yanai said that “at the root of the situation is that science is not seen by administrators and funding agencies as a fundamentally creative process.” Instead, he said, it’s “portrayed as only hypothesis-testing, which is just half of the method.”

If institutions were to “recognize more fully that the main engine of science is coming up with creative new ideas in the first place,” Yanai said, “then we would ensure that scientists have more protected time.” To that end, Yanai and a colleague have been working a project called Night Science, which includes editorials and a podcast on the scientific creative process.

Writing about the new Nature study via Substack, political scientist Anna Meier compared “grant culture,” or the endless pursuit of funding to do one’s research, to “the grim reaper.”

When “you have to get a grant and competition for grants is so extensive, you propose the project that is most likely to get funded. Proposing something off the beaten path is an enormous risk in time and energy potentially wasted. This is especially true if it’s your first grant application, because getting one grant opens the door to getting others,” Meier wrote. “I think a lot of us are scared shitless of the grant grim reaper.”

The solution, Meier continued, “isn’t more ‘how to get a grant’ panels or emails with application opportunities. Those tactics may help on the margins, but they also overwhelm, and their functionalist framing inspires resentment rather than empowerment. An actual answer is structural transformation, starting (perhaps paradoxically) from the state with a recommitment to proper funding for higher education and the research its highly trained experts know how to prioritize and execute, if they are only given the space to do so.”

Drawing on her own experiences seeking funding from the National Institutes of Health, neuroscientist Elaine Sevier wrote in a similar essay for Research Theory that “scientists, generally, work very hard for relatively little money because they genuinely want to make discoveries. Everyone wants to write the classic paper that is still referenced in 100 years. But to advance your career, you need an easily digestible story that will be immediately popular—and you should seek, interpret, and present your data accordingly. This is a constant cycle of short-term thinking that does not tolerate slow, sometimes boring, sometimes weird work that lies outside the mainstream. That also means it does not tolerate the work that has the potential to become disruptive.”