Making Better Ed-Tech Buying Decisions

The University of Virginia’s Curry School of Education launched the Jefferson Education Accelerator in early 2015, with a goal of helping growth-stage ed-tech companies compete for university eyeballs with their more established counterparts. After modest returns and sizable roadblocks, though, the accelerator’s focus has drifted toward addressing systemic issues in the broader ed-tech landscape.

“As we’ve started doing that work, we’ve just kept running into problem after problem, seeing that these companies had a hard time competing on the basis of merit, not marketing,” Bart Epstein, the accelerator’s CEO, told “Inside Digital Learning.”

College administrators and faculty members have proved even more reticent than expected to embrace new ed-tech providers, Epstein said. Even with endorsements and support from the accelerator, which aims to eventually become a Consumer Reports for all things ed tech, many prefer to avoid risks that come with banking on unfamiliar technologies and companies.

In general, university decision makers are overcommitted and underinformed about the technology providers and tech tools they encounter. Those factors sometimes make the prospect of switching midstream to a new ed-tech provider daunting, leaving new companies struggling to compete with established brands, Epstein said.

Challenges Abound

Early on, the accelerator encouraged its tech company "members" to emphasize to universities the merits of their products, which officials said in many cases were more innovative than those of the more established ed-tech providers. But discussions of specifications frequently fell on deaf ears, as university decision makers often lack the time and energy to weigh specific merits against other factors.

“When people are overwhelmed and they have to make decisions, it’s understandable for them to care about things like dealing with a known entity,” Epstein said.

Successes weren’t completely elusive. The accelerator’s first member was the lecture-capture company Echo360, which has used Jefferson’s tools to bolster research indicating that students who use its active-learning program are more engaged than those who don’t. In general, according to Epstein, the accelerator’s primary strength thus far has been as a research partner, though marketing help hasn’t entirely been a flop.

“None of the companies that we have worked with have failed,” Epstein said. “All of the companies that we worked with, we still have the equity and we’re still supporting them and hoping that their performance will yield good financial and academic outcomes.”

Epstein declined to offer specifics on the number of companies the accelerator has worked with. Companies pay the accelerator with equity, typically between $200,000 and $300,000.

A Wider Scope

When the accelerator’s original mission flagged, Epstein directed his team to look outward: what’s keeping universities from buying technologies that will almost certainly benefit them in the long run, and what’s keeping ed-tech companies from making a convincing enough cases about those benefits?

Interviews with stakeholders -- entrepreneurs, educators, policy makers -- revealed a few key themes.

1. All agreed that ed-tech decisions should be made based on evidence of what’s most likely to work.
2. All agreed that decisions are often not being made based on those criteria.
3. None agreed on who’s to blame for the above.

“They all pointed fingers,” Epstein said.

Educators had numerous concerns and gripes, according to Epstein. Many administrators don’t believe their occupational duties should include making technical decisions about which they often haven’t been trained. University leaders can usually only prioritize one or two initiatives at a time, leaving new technologies to fall by the wayside. Online reviews or other resources don’t provide enough context about how a product could be beneficial or how it compares to others on the market.

Epstein and his team think these disparities represent a major impediment in the evolution of the ed-tech landscape. They’ve spent much of 2017 convening stakeholders for a series of working groups to address what Epstein calls “big, structural problems” in the space.

The working groups span a broad range of topics for higher education as well as K-12. They include “Institutional Competence in Evaluating Efficacy Research,” “Crowdsourcing Efficacy Research and Product Reviews” and “Research Spending and the Most Popular Ed-Tech Products.” The early fruits of those efforts were unveiled in Washington at a research symposium in May.

Fiona Hollands, associate director and senior research for the Center for Benefit-Cost Studies of Education at Teachers College at Columbia University, led the group that sought to reveal and analyze the decision-making process at the university level. Her team discovered a paradox.

“On the one hand, they complain there isn’t enough research out there about ed tech,” Hollands said of higher ed decision makers. “On the other hand, they say about the research, ‘I don’t really find it that relevant anyway.’”

Instructors want “local evidence” of an ed-tech provider’s success and viability before committing to it at their institution. Research conducted halfway across the country or at a much bigger or smaller institution rarely suffices. Proper studies with a control group are often difficult to conduct because students sometimes balk at one portion of their class getting to use new equipment while the other portion can’t.

Looking Ahead

The report from Hollands’s group recommends a wide range of techniques for resolving ed-tech tensions, including multisite pilots with other universities, involving faculty members and students earlier in the process, and streamlining and standardizing the procurement process.

Some observers expect these solutions to take a while to pan out. Casey Green, founding director of the Campus Computing Project, which details the role of information technology in American higher education, believes institutions often make technology decisions based on “opinion and epiphany,” and convincing them to do otherwise remains a long-term project. Companies hoping to make a splash might simply have to wait longer to gather momentum, said Green, an Inside Higher Ed blog contributor.

“The resources have to capture their own credibility and gain their own traction,” Green said. “You have to earn this. You don’t get it just by declaring that you’ve got it.”

Epstein envisions a future in which ed-tech products come equipped with “minimum system requirements” akin to listings on software, and in which a data hub collects research findings from disparate research groups studying the impacts of ed tech. He also wants to see more collective action among stakeholders to proactively address impediments to ed-tech evolution.

All of these goals will be part of the Jefferson accelerator’s mission going forward, according to Epstein, bolstered by support from numerous philanthropic foundations and organizations including Strada Education Network, the Bill and Melinda Gates Foundation, CZI, Overdeck, Kauffman Foundation, New Schools, and the Michael and Susan Dell Foundation.

“We are in the very, very early stages of collectively thinking about that, and I think there’s growing momentum for it,” Epstein said. “It’ll take a lot of collaboration, but we’re moving inexorably toward more information and more transparency.”