LLM cost accounting · defect note

Streaming message_delta erases accumulated usage

Your streamed calls report less than they cost and your non-streamed calls are perfect. That asymmetry has one common cause: the final event was assigned to your usage variable instead of added to it, discarding everything counted before it.

How usage arrives on a stream

A streamed Anthropic response spreads usage across events rather than handing you one object at the end. message_start carries the input side — including the cache fields. message_delta arrives last and carries the output side.

event: message_start
data: {"message": {"usage": {
  "input_tokens": 25,
  "cache_read_input_tokens": 75,
  "output_tokens": 1
}}}

event: message_delta
data: {"usage": {"output_tokens": 412}}

Note what the final event does not contain: input_tokens, and neither cache field. It is a delta, not a summary. Anything that treats it as a summary loses the input side entirely.

The bug

# WRONG — the last event replaces everything counted before it
async for event in stream:
    if event.usage:
        usage = event.usage        # input_tokens and cache reads: gone

# RIGHT — the input side is captured once, the output side accumulates
async for event in stream:
    if event.type == "message_start":
        usage = event.message.usage
    elif event.type == "message_delta":
        usage.output_tokens = event.usage.output_tokens

In the wrong version the recorded input is whatever the final delta happened to carry, which is usually nothing at all. On a cache-heavy agent workload — where the input side is the expensive side — this discards the majority of the bill. I fixed this in mcp-use #2127, alongside uncounted Anthropic cache tokens in the same code path.

Why it survives review

The code reads correctly. if event.usage: usage = event.usage looks like defensive assignment — take the usage whenever the event has usage. It is only wrong because of a protocol detail that is not visible at the call site: that later events carry a subset of the fields, not a superset.

Nothing raises. The stream completes, the text is correct, the usage object exists and has plausible integers in it. The only observable symptom is a number on a dashboard that is too small, months later.

The test that catches it

Assert against a recorded event sequence rather than a live call, so the test is deterministic and runs in CI:

def test_streamed_usage_keeps_the_input_side():
    events = [
        message_start(input_tokens=25, cache_read_input_tokens=75, output_tokens=1),
        content_delta("..."),
        message_delta(output_tokens=412),
    ]

    usage = collect_usage(events)

    # the input side survives the final delta
    assert usage.input_tokens == 25
    assert usage.cache_read_input_tokens == 75
    # and the output side is the final value, not the first
    assert usage.output_tokens == 412

The first two assertions fail on the buggy implementation and pass on the fixed one. That is the whole test. Every fix I ship carries one like it, because a correctness fix without a test that fails on main is just a diff someone will undo.

Checking your own stack

1. Split one production day's reconciliation by stream=True and stream=False.
2. If the gap sits almost entirely on the streamed side, this is your defect.
3. Find every handler that reads usage from a stream event and check whether it assigns or accumulates.
4. Check gateways too — a proxy that re-emits events can drop message_start fields on the way through.

Related: OpenAI prompt_tokens vs Anthropic input_tokens · Why your Claude bill is higher than your dashboard

← Back to the audit