Redefining Impact: Why EdTech Must Move Beyond Proof of Promise

For more than a decade, EdTech has promised impact. And to be fair, there has been progress: more data, more dashboards, more case studies, more charts pointing up and to the right. But here’s the uncomfortable truth:

The bar for “impact” in EdTech has quietly collapsed.

Today, impact is too often declared through polished narratives rather than proven through evidence. Descriptive trends are treated as outcomes. Averages are used to mask meaningful variation across classrooms. Positive results are celebrated without a single test of statistical or instructional significance. Anecdotes are mistaken for evidence. Marketing assets are passed off as proof.

And districts, under immense pressure to accelerate learning, are left to sort out the difference.

This series exists because that is no longer acceptable.

The problem isn’t data. It’s discipline.

The EdTech industry doesn’t suffer from a lack of data. It suffers from a lack of discipline in how that data is interpreted, tested, and communicated.

We’ve normalized claims like:

• “Teachers loved it”
  
• “Scores went up”
  
• “Early results are promising”
  
• “The average student improved”
  

Without asking harder questions:

• Were the gains statistically meaningful?
  
• Who benefited and who didn’t?
  
• Did the results persist beyond the initial implementation bump?
  
• Can the outcomes be attributed to instruction, or are they just noise?

When everything is labeled “impact,” the word stops meaning anything at all.

Why positive results are no longer enough

Positive outcomes are easy to find in complex systems like education. Variability alone guarantees that some scores will rise somewhere, sometime.

That’s why real impact has never been about positivity. It has always been about:

• Signal over noise
  
• Attribution over correlation
  
• Durability over novelty
  
• Equity over averages
  

Yet much of the market still rewards the appearance of success over its validity.

District leaders deserve better. Teachers deserve better. And students deserve far better than decisions based on fragile evidence.

A market at an inflection point

We are entering a new phase of EdTech maturity.

Budgets are tighter. Renewals are harder. And districts are asking more sophisticated questions. Not just Did it work? but Will it work here, for our students, at scale, over time?

This marks a shift from proof of promise to proof of performance. And that shift demands a higher standard of impact.

Introducing The Impact Reality Series

This series is a deliberate response to the widening gap between how impact is claimed and how impact should be evaluated.

Each post challenges a widely accepted, but rarely examined, assumption in EdTech impact discourse:

• That what’s labeled “impact” is actually evidence, not marketing
  
• That averages tell a complete or honest story
  
• That positive trends are the same as meaningful learning
  
• That meeting minimum evidence standards is sufficient
  
• That early gains automatically endure 

Together, these posts make a single argument:

Impact is not a moment. It is a system. And systems demand rigor.

What this series is and is not

This is not a critique for critique’s sake. It is not a rejection of innovation, experimentation, or iteration. It is a call for clarity.

A push to distinguish:

• Measurement from marketing
  
• Evidence from anecdotes
  
• Outcomes from optics

And to equip district leaders with the language and frameworks needed to evaluate impact claims with confidence. When standards rise, everyone benefits, especially students.

A commitment, not a campaign

Publishing this series is a commitment:

• To transparency over theatrics
  
• To instructional truth over easy narratives
  
• To impact that holds up not just in presentations, but in practice

At Kiddom, we believe impact should be earned, not asserted. That belief shapes how we build curriculum, technology, professional learning, and evidence, treating impact not as a one-time study or static asset, but as a living system that must be validated, sustained, and refined over time.

This series reflects that conviction and our belief that the future of EdTech belongs to technologies willing to hold themselves to a higher standard.

One thing you can do right now, with lasting impact:

Before reviewing any vendor evidence, align internally on what counts as impact. 

Create a shared definition with your leadership team, one that includes statistical validity, instructional attribution, equity, and durability, so every tool is evaluated against the same standard.

Next up: Impact or Illusion? Why EdTech Must Stop Marketing “Impact”

Before we can talk about results, we have to talk about truth. District leaders are being asked to make million-dollar decisions based on one-pagers, pilot decks, and polished narratives. Post 1 challenges the most common sleight of hand in EdTech impact claims, mistaking marketing assets for evidence, and clarifies what real proof of learning actually requires.