LLM cost accounting · defect note

Why your Claude bill is higher than your dashboard

Your application says you spent $4,100 on Anthropic last month. The invoice says $11,800. Nothing errored, no test went red, and both numbers look plausible. Here are the four causes, ranked by how often each one turns out to be the culprit.

1. Cached tokens were never counted

This is the overwhelming favourite. Anthropic reports cache activity in fields that sit outside input_tokens:

"usage": {
  "input_tokens": 25,
  "cache_read_input_tokens": 75,
  "cache_creation_input_tokens": 0,
  "output_tokens": 40
}

A library that normalises providers and writes total = input_tokens + output_tokens records 65 tokens for a request that actually consumed 140. You are billed for the cache read. Your dashboard is not.

The tell: the gap scales with your cache-hit rate. If the discrepancy grows as caching improves, this is your bug. I have found and fixed exactly this in Pipecat, LiveKit, mcp-use, deepset Haystack and Meta's Llama Stack. Full write-up on the cache-token defect →

2. The pricing table is stale or missing a model

Token counts are right; the multiplier is wrong. A model ships, your pricing map doesn't have it, and the lookup quietly falls back to zero or to a sibling model's rate. In Pydantic's genai-prices I found missing Bedrock entries for Writer Palmyra X4 and X5 — those calls priced at zero. Not approximately zero. Zero.

The tell: the gap is flat regardless of caching, and it often appears as a step change on the date a model was adopted. Check whether any model in your logs is absent from your price map, and whether any request costed exactly 0.

3. Streaming lost the usage it had already accumulated

On streamed responses, usage arrives across events. Code accumulates it per chunk, then a final message_delta arrives and the handler assigns rather than adds — erasing everything counted so far.

# wrong: the final delta replaces the running total
usage = event.usage

# right: the final delta contributes to it
usage.output_tokens += event.usage.output_tokens

The tell: streamed calls under-report and non-streamed calls are correct. Split your reconciliation by stream=True and see whether the gap lives entirely on one side. This was part of mcp-use #2127.

4. Failed and retried calls are billed but not recorded

A request that times out client-side, or throws while you are parsing its response, was still served and still billed. If your usage recording happens after a successful parse, every failure is invisible spend. Retries double this: three attempts, three charges, one record.

Worse, usage tracking can fail the very call it is measuring — I fixed that pattern in Roboflow's inference package, where a tracking error propagated into the request path.

The tell: the gap correlates with your error and retry rate, not your cache-hit rate, and it spikes during incidents.

The ten-minute check

1. Pick one production day. Sum the cost your system reported for Anthropic calls.
2. Take the actual figure for the same window from the Anthropic console.
3. Compute the gap as a percentage. Under 2% is rounding. Over 10% is a defect.
4. Split the same day four ways — by cache-hit rate, by model, by stream=True, and by error rate. Whichever split makes the gap move is your cause.

That last step is the whole method. A single aggregate number tells you that something is wrong; the splits tell you which of the four it is. Most teams never run the splits, because nothing has broken loudly enough to make anyone look.

Why nobody catches this

None of these four raise. The call succeeds, the response is valid, the number is plausible, and no test asserts that reported spend equals billed spend — because writing that test means reconciling against an invoice, which nothing in a normal CI pipeline can reach. So the defect survives until someone reads a bill and frowns.

Related: Anthropic cache tokens are not in total_tokens · OpenAI prompt_tokens vs Anthropic input_tokens

← Back to the audit