The OpenTelemetry Gotcha That Multiplies Your Metrics (Simply Explained)

The number that looked right but wasn't

I had a dashboard tracking my AI assistants. I run a bunch of them, and each one costs money to operate. So I built a screen to show me how much work each one was doing and how much it was costing me.

One of my assistants showed a usage number of about 120,000. That seemed believable. But I happened to know the real number was around 30,000.

The dashboard had quietly multiplied the truth by four.

No error message. No crash. No red warning anywhere. Just a clean, professional-looking screen telling me a confident lie.

That's the most dangerous kind of mistake. If a number is missing, you know something is wrong. But a wrong number that looks reasonable? You believe it. You make decisions on it. And every decision after that is built on sand.

Why the numbers got multiplied

Here's the simplest way I can explain what went wrong.

Imagine you're tracking a runner during a race. Every five seconds, someone shouts out the total distance she's run so far. Not the distance since the last shout. The total from the start.

So you hear: 100 meters, then 250, then 400, then 600, all the way up to the finish at 1,000 meters.

The last number, 1,000, is the real answer. That's how far she ran. Every shout before that was just a snapshot of the same run, partway through.

My system was collecting all those shouts and adding them together. So instead of recording 1,000 meters, it added up 100 plus 250 plus 400 plus 600 and so on. The total came out massively bigger than the truth.

That's exactly what happened with my AI usage numbers. The tool reports a running total every few seconds. My software was treating each report as a brand new event and stacking them on top of each other.

The longer an assistant ran, the more reports it sent, and the more inflated its number got. So my busiest, most important assistants had the most distorted numbers. The ones I cared about most were the ones lying to me hardest.

The two-line fix

Once I understood it, the fix was almost embarrassingly simple.

Since each report is just the running total, and that total only ever goes up, the correct answer is just the highest number you saw. Not the sum. The biggest one.

Going back to the runner: you don't add up every distance someone shouted. You just take the final, biggest number. That's how far she ran.

So I changed my system to keep the highest value it had seen for each assistant, instead of adding them all together. The number that said 30,000 stayed at 30,000. Done.

There's a bonus to doing it this way. These tracking systems are messy. They sometimes send the same report twice, or send them out of order, or resend old ones. With the old "add them up" approach, every duplicate made my numbers worse.

With the "keep the highest" approach, duplicates do nothing. If a report comes in saying 18,000 but I've already recorded 30,000, it just gets ignored. The worst a glitch can do is nothing at all. That's the difference between a system that's right on a good day and one that's right every day, including the messy ones real life throws at you.

Why this kind of bug is the scary one

Step back from the technical stuff for a second, because this is the real lesson.

This bug never crashed anything. It never showed a warning. It just produced wrong numbers that looked completely believable, on a dashboard that looked finished and trustworthy.

That's the worst kind of failure. A crash gets fixed because it screams at you. A wrong number that looks right gets used.

If I'd taken those inflated numbers and reported my costs, I'd have thought my AI assistants were four times more expensive than they actually were. If I'd ranked them by how much work they did, the chattiest ones would have looked the most productive, purely because they reported more often. Every conclusion would have been garbage, and I'd have had no reason to doubt a single one.

I've seen the opposite version of this too, where a system showed zeros for two weeks and everyone just assumed it was a quiet stretch. Wrong-but-high and wrong-but-zero are cousins. Both are your tools lying to your face with a straight expression.

The thing that catches this isn't fancier technology. It's understanding how the data underneath actually works. Knowing that these reports are running totals, not separate events. Knowing the small details that decide whether your numbers mean anything at all.

Who you let near your numbers matters

Here's the part that should matter to you as a business owner.

Anyone can throw together a dashboard these days. You can ask AI to build you one and it'll run. It'll show numbers. It'll look professional.

Whether those numbers are correct is a completely different question. And the demo doesn't answer it.

Knowing that you take the highest value, not the sum. Knowing that duplicate reports shouldn't corrupt your totals. That's the gap between something that looks like a real system and something you can actually trust to make a decision.

That's the work I do. I build the systems, and I understand the layer underneath them. When numbers drive decisions, I want solid, predictable code owning the math, not a guess.

I've built more than 15 of these AI systems across my own fashion brand and for clients. The reason they hold up is that I read how the underlying tools actually behave instead of assuming. The cost of assuming isn't a dramatic crash. It's two weeks of confident decisions built on a number that was four times too big.

If your team is building anything where the output shapes what you do next, you want someone who checks the details before the wrong numbers end up in a board meeting.

Ready to bring AI leadership into your company?

I work with a small number of companies at a time. If you're serious about AI, apply to work together and I'll review your application personally.

Apply to Work Together