We Need to Ask Smarter Questions of Ed Tech

The platforms I once promoted could show you a dashboard of open rates and response times. What they couldn’t show you was whether a student felt heard—or safe enough to speak.

I used to celebrate response rates north of 80 percent on messaging campaigns aimed at enrollment or advising milestones. Now, as a professor, I find myself wondering, what about the students who didn’t respond—and what assumptions were baked into the tech that tried to reach them?

Before becoming a tenure-track assistant professor of organizational communication, I spent more than a decade working in the educational technology industry, ultimately serving as vice president of marketing and communications for three companies. During that time, I collaborated on communication tools designed to support students across their journey, from recruitment to advancement, and helped institutions adopt those tools in order to scale student support and boost engagement.

After years in industry, I returned to academia to better understand the systems I once sold, earning a doctorate in educational technology focused on how communication unfolds in tech-mediated learning environments. Postgraduation, I transitioned to academia full-time, and now one of my research programs is examining how the Silicon Valley mindset—obsessed with speed, disruption and scale—is increasingly shaping communication practices in nontech sectors like higher ed.

As someone who marketed under the influence of this logic for years, my research is now making me question how it functions in contexts where relationships, not growth, should come first.

Communication is the thread that runs through all my work: I’ve marketed it, I teach it and I study how it functions in digital learning environments. For my entire professional life, one central idea has remained consistent: Technology can serve as a bridge between institutions and learners. From my current vantage point in the classroom, I still believe in that idea’s potential. But I’ve also seen just how far its execution can fall short. With this new, in-the-classroom perspective, I view ed tech’s promises through a more critical and, frankly, more complicated lens.

That’s not to point fingers. Even with a decade spent helping ed-tech firms scale, I’ve fallen into the same trap that catches many of my colleagues. I’ve built assignments assuming students would find certain tools intuitive or engaging, only to discover that I was projecting my own industry familiarity onto students who were confused, hesitant or simply disengaged. Those moments have been humbling. They remind me that the gap between technological promise and student experience is often wider than we think—and that gap has only grown with the rise of generative AI.

With that in mind, this is my call to action for faculty, academic leaders and anyone involved in evaluating or adopting ed-tech tools on campus. From my dual vantage point as a former ed-tech executive and now a faculty member, I’ve come to believe that we need to ask smarter questions about the claims we accept, the systems we adopt and the kind of culture we create through communication.

Ed Tech Shouldn’t Be Exempt From Scrutiny—So Why Is It?

In academia, we scrutinize syllabi, peer-reviewed articles and course learning outcomes. But when it comes to ed tech, and especially communication platforms, we often accept the sales pitch without applying the same level of rigor we bring to every other part of our professional lives.

The generative AI gold rush has made things worse. Instead of thoughtful debate, we’re rushing full speed into binary extremes. AI is a savior or a scourge. Faculty are Luddites or evangelists. Institutions either embrace automation or cling to analog systems that no longer scale.

This kind of thinking isn’t just exhausting. It’s dangerous. It discourages nuance, hinders policy development and leads to reactive decisions that ultimately hurt our students.

I’ve seen institutions struggle at both extremes. Some lean heavily into automation, delegating to AI everything from enrollment messaging to wellness check-ins. Others delay decisions for fear of getting it wrong. Both approaches can leave students underserved.

What We Expect From Research, We Should Expect From Ed Tech

I understand the benefits automation can offer. AI tools can reduce lag time and improve consistency. But they can’t model care. They can’t detect when a student is anxious before an exam or too ashamed to ask for help. These moments often shape a student’s trajectory, and they require more than just an algorithm.

When we rely too heavily on automation, we lose something essential: students’ sense that someone actually sees and hears them. When we start believing that faster communication is better communication, we risk mistaking automation for empathy.

We would never publish a paper that cherry-picks findings, ignores methodological limitations or presents correlation as causation without rigorous justification. Rigor, replicability, transparency—these standards shouldn’t apply only to scholarship. They must also apply to any system claiming to enhance educational outcomes.

To avoid falling for tech that ultimately distances us from our students, I propose that we adopt shared definitions of reliability and validity in ed-tech marketing content, and hold the industry more accountable.

Reliability in Ed-Tech Marketing Content

Reliability in this context refers to how consistently results can be replicated across comparable contexts. Many ed-tech marketing materials don’t meet this standard. They report outcomes from a single institution without showing whether those results are representative or exceptional.

To assess reliability, ask,

• How many other institutions of my size have seen similar results?
• How many others of my type?
• What about in my region?
• Or with a similar student body composition?

If a company can’t name multiple examples that match on these dimensions, we should treat that absence as a meaningful data point, even if it doesn’t appear in the brochure.

Validity in Ed-Tech Marketing Content

Validity asks whether a claim accurately represents what it’s supposed to.

Some ed-tech claims are reasonably valid, especially those grounded in transparent methodology or developed by people with academic research experience. But others oversimplify. They jump from a tool’s feature to broad impacts on retention, engagement or graduation rates, without acknowledging other variables at play.

To probe validity, ask,

• How was this data collected? Was it primary or secondary?
• Does the suggested correlation make sense? Does what the product does plausibly lead to the claimed outcome?
• If the connection is distant or indirect, what other factors might explain the result? Are they acknowledged or conveniently ignored?

Whether companies genuinely believe their claims or simply hope buyers won’t ask too many questions, we do our institutions a disservice if we accept these narratives at face value. It’s on us to read between the lines.

Don’t get me wrong—we don’t need peer-reviewed journal standards for marketing teams. But we should expect more from materials designed to influence multiyear, often multimillion-dollar decisions. Reliability and validity should be the starting point, not an afterthought. Asking the kinds of questions outlined here helps ensure ed tech supports, rather than undermines, our mission to serve students well.

And with that goal in mind, here are some other, tougher questions we can ask ed-tech vendors early and often in the sales process.

As an ed-tech decision-maker, your first instinct might be “I don’t need the salesperson to explain the tech. I’m smart. I can read about it myself.” But this question isn’t about proving your expertise. It’s about testing theirs. If someone is trying to sell you an ed-tech platform with bells and whistles, they should know how the bells chime and why the whistles matter. A reasonable answer for something like “what is machine learning” might be “It’s a subfield of AI. With machine learning, systems can ‘learn,’ or improve from training without being explicitly programmed.”

Understanding the tech at a surface level is a start. But smart buyers dig deeper. What does machine learning mean at this specific company? A reasonable response might be “Machine learning is integrated to save our clients time. For instance, it’s used for generating FAQs.”

Understanding how the technology shows up in a project today is critical. Ed-tech companies are often built on big visions, but you’re paying for what exists now, not what’s on the road map. How does machine learning show up in this chat bot today? A solid response might be “Today, [specific features of our chat bot] can use machine learning, but there’s an additional fee to activate these features.”

Are You Convinced?

Marketing materials, whether websites, ads or outreach emails, may use the same buzzwords to justify high price tags. But don’t assume that every company fully understands, integrates or delivers on what they sell. An open, informed conversation is the first step toward ensuring that a product truly does what it claims.

Think about it in another way: We wouldn’t assign a textbook with unverifiable claims. Why tolerate that from ed tech?