Sitemap

When should data teams worry about platform costs? đź’¸

10 min readDec 30, 2024

I generally find that the way data grows and the way we price its storage and computation follow entirely different trajectories relative to business growth. Businesses don’t pay a linear fee for more insight and instead pay an exponential tax. If revenue is growing by x, your data is probably doubling at 2x, and the cost to handle it might be ballooning by 3x or 4x. That’s pretty much the arithmetic I’ve seen in any Data Platform or Infra orgs.

Every query, every event produced, every real-time dashboard, every machine learning model used to power recommendations for customers will chip away at profitability. You’ll tell yourself that you created a “machine good at printing money”, but it’s also good at printing a giant electricity bill.

Press enter or click to view image in full size

And while everyone acknowledges at some level that growth in data costs outpaces business growth, few actually pause to examine the mechanics. What causes this growth? Why do costs snowball? And, most importantly, what should we be doing about it?

Less is more

Many teams assume, incorrectly, that you can “optimize” your way out of a data cost spiral . That the right tool or vendor can flip a switch and make the problem disappear. My experience tells a different story. Unless you’re deliberate about how you manage access policies…

--

--

Responses (2)