LLM cost accounting · defect note

Anthropic cache tokens are not in total_tokens

If your Anthropic invoice is higher than the number on your own dashboard, the most likely cause is not sampling, retries or a pricing table. It is that your cached tokens were never counted.

The shape of the bug

Anthropic's usage object reports cache activity in fields that sit outside input_tokens:

"usage": {
  "input_tokens": 25,
  "cache_read_input_tokens": 75,
  "cache_creation_input_tokens": 0,
  "output_tokens": 40
}

A request that actually consumed 100 tokens of input reports input_tokens: 25. That is correct, and it is documented. The defect appears in the layer above, when a library normalises providers into one shape and writes:

total_tokens = usage.input_tokens + usage.output_tokens   # 65, not 140

Nothing raises. The call succeeds, the response is valid, the number is plausible, and every cost figure computed downstream is low by exactly the cached portion. On a workload built around prompt caching — which is to say, most agent workloads — that portion is the majority of your input.

Both directions are real

Under-counting is the common case, but the inverse ships too. In Meta's Llama Stack the Anthropic-compatible endpoint copied OpenAI's inclusive prompt tokens into Anthropic's exclusive input tokens and added the cache read on top. A 100-token request with 75 cached totalled 175. Same root cause, opposite sign, and equally silent.

In a mixture-of-agents aggregator at Nous Research, an auxiliary usage shim dropped cache_read and cache_creation before the aggregation step. Cost under-reported by roughly 7×.

Where it hides

• Provider adapters. Any function normalising several providers into one usage shape. This is where the inclusive/exclusive mismatch lives.
• Streaming paths. Usage accumulated across chunks is frequently overwritten by a final message_delta, erasing what was already counted.
• Gateways and routers. Anthropic models served through an OpenAI-compatible proxy lose cache fields at the translation boundary, because the OpenAI schema has nowhere to put them.
• Bedrock and Vertex. The same model behind a different vendor's envelope often uses a third field naming convention again.
• Observability exporters. Cost can be correct in your application and wrong on the dashboard, if the span attributes drop the cache breakdown in transit.

Check your own stack in ten minutes

1. Pick one production day. Sum the cost your system reported for Anthropic calls.
2. Open the Anthropic console and take the actual spend for the same window.
3. If your number is low, log one raw usage object and confirm whether cache_read_input_tokens is non-zero and whether it reaches your total.
4. Grep your codebase for input_tokens and check every site that adds it to output_tokens without also reading the two cache fields.

A gap that tracks your cache-hit rate is this bug. A gap that is flat regardless of caching is a pricing table problem instead, which is a different and easier fix.

Prior fixes

I have found and fixed this defect class in production libraries, each shipped with a test that fails on main: Pipecat (Anthropic and Bedrock services), LiveKit agents (Bedrock prompt_tokens), mcp-use (uncounted cache tokens plus a streaming message_delta erasing accumulated usage), deepset Haystack (the OpenAI-compatible integration), ogx — formerly Meta's Llama Stack — and Roboflow, where a usage-tracking failure could fail the very call it was measuring. In the OpenAI Agents SDK, every responses.compact call was billed but left out of run totals; I wrote the first fix and an OpenAI engineer's superseding patch shipped with my credit as co-author.

Related: OpenAI prompt_tokens vs Anthropic input_tokens · Streaming message_delta erases accumulated usage · Why your Claude bill is higher than your dashboard

← Back to the audit