# Three metering bugs that turn token invoices into disputes

> Two of them can be designed out of the usage event from the start; the third needs reconciliation finished before the invoice is finalized.

Original: https://fdetimes.net/en/guides/token-metering-bugs-billing-disputes/

A customer emails a CSV file. Their logs show one token count, this month's invoice shows another, and they want to know why. At that point the engineer who built the integration is answering a question about money, and the answer has to match line by line.

No major incident is needed for this to happen. Three small flaws baked into the design are enough: undercounting because a usage field was misread, double counting because of retries, and late counting because events arrived after the billing period closed.

For an FDE deploying an AI product billed by the token, all three belong to engineering. The finance team cannot fix them.

Each one can be prevented in advance, on one condition: every model call is treated as an accounting entry, with the rigor that implies, rather than as one more line in a log.

## Is input_tokens the total input?

No, at least with Anthropic. Anthropic's prompt caching documentation states that the `input_tokens` field counts only the tokens after the last cache breakpoint, which excludes much of what was actually sent. Total input is `cache_read_input_tokens + cache_creation_input_tokens + input_tokens`.

Take an agent with a long system prompt that has already been cached. A request returns `input_tokens = 200`, `cache_read_input_tokens = 8000`, `cache_creation_input_tokens = 0` and 500 output tokens. A metering system that reads only `input_tokens` records 200 input tokens. The real figure is 8,200.

It gets harder because providers do not count the same way. The documentation for the ReqLLM library notes that Anthropic usage typically carries the flag `input_includes_cached: false`, while OpenAI's is typically `true`. Applying the three-field sum to a provider that already folds cache into input counts the cache twice.

Even with a correct total, a single aggregated number should not be sent. ReqLLM points out that cache reads and cache writes have separate counters because providers may price them differently, and reasoning tokens may carry their own price too. If the customer contract has several price tiers, the event has to keep each token type separate.

## Sent twice, billed once

The second bug lives in the network layer. Usage goes to the billing system, the request times out, the code retries, and the first attempt had in fact succeeded. Stripe recommends idempotency keys for precisely this case: every meter event has an identifier that can be set by the caller, and if it is left blank, Stripe generates one.

The trap is in "Stripe generates one". If each retry lets Stripe create a fresh identifier, the two sends become two distinct events. The identifier must be deterministic, meaning it can be derived from the model call itself, for example by combining the customer ID with the provider's request id.

Lago's write-up of its ingestion architecture defines exactly-once along the same lines: deduplication based on `event_id` and an `idempotency_key` generated on the client side. The function below does both jobs, normalizing usage and attaching the identifier:

```python
def build_usage_event(provider, customer_id, request_id, usage, ts):
    # usage has already been mapped to common field names; original names vary by provider
    cache_read = usage.get("cache_read", 0)
    cache_write = usage.get("cache_write", 0)
    raw_input = usage["input"]

    INPUT_INCLUDES_CACHED = {"anthropic": False, "openai": True}
    if INPUT_INCLUDES_CACHED[provider]:
        uncached_input = raw_input - cache_read - cache_write
    else:
        uncached_input = raw_input

    return {
        "identifier": f"{customer_id}:{provider}:{request_id}",
        "customer_id": customer_id,
        "timestamp": ts,  # time of the model call, not time of sending
        "uncached_input": uncached_input,
        "cache_read": cache_read,
        "cache_write": cache_write,
        "output": usage["output"],
    }
```

Call this function twice for the same request and it returns the same identifier. That is the whole deduplication mechanism: the downstream system sees the event already exists and discards the second send.

Note that the `timestamp` field is taken from the time of the model call. Stripe's documentation requires timestamps to fall within the past 35 calendar days and no more than 5 minutes in the future. A server clock running a few minutes fast, or a backfill job that runs too late, is enough to get events rejected.

## At high volume, aggregate before sending

Stripe limits Meter Events to 1,000 calls per second per account, and each customer-and-meter pair is allowed only one concurrent call. An agent running many parallel tasks for the same customer will hit the second limit very quickly.

The sensible approach is to write every call to an internal ledger, then aggregate by time window before sending. The identifier must then be deterministic per window, for example `customer:meter:2026-11-17T10:05`, so that resending a window can never produce two events.

The cost is a decision about when to close a window. Events that arrive after a window has been sent go into a new window. Editing the old one would break the deduplication mechanism you built.

**Key point:** Your ledger is the source of truth; the billing system is a copy that needs checking.

## Reconciliation must finish before the invoice locks

The third bug is the hardest to spot because nothing breaks right away. Stripe processes meter events asynchronously, so the usage totals in the meter summary and on the upcoming invoice may not yet include recently sent events. Invalid events are also reported only afterwards, through error Events that you must listen for, fix and resend.

The time available for all this is limited. By default each Stripe invoice has a 1-hour grace period before finalization, adjustable up to 72 hours, during which usage can still be reported for the previous period. Usage that arrives after finalization still appears in the meter summary but is added neither to the finalized invoice nor to the next one.

That is the situation that produces a dispute: the dashboard shows one number, the invoice another. Lago also observes that late-arriving events break reconciliation. Reconciliation therefore needs to run on a fixed schedule, well before any customer asks.

The minimum schedule has four steps. First, compare the internal ledger totals with the meter summary every day, broken down by customer and by token type. Second, for each discrepancy, trace down to individual identifiers to find out whether the event was never sent, failed, or is still pending.

Third, read the error Events Stripe returns, fix the broken events and resend them with the same original identifier. Fourth, when the period ends, rerun the comparison inside the grace period and send any missing usage before the invoice locks.

| Symptom in reconciliation | Common cause | What to do |
|---|---|---|
| Ledger higher than meter, consistently on cached requests | Reading only `input_tokens` | Fix normalization, send the shortfall within the period |
| Meter higher than ledger | Retries generating new identifiers | Switch to deterministic identifiers |
| A few events off, scattered | Asynchronous errors or timestamps outside the allowed range | Read error Events, fix and resend |
| Discrepancy concentrated at period end | Sent after the invoice was finalized | Extend the grace period, reconcile earlier |

## Mistakes FDEs commonly make

The most common is trusting that `input_tokens` is the total, because the name suggests it. The next is using the send time instead of the model call time, which pushes calls made just before the period cutoff into the following period. The last is letting the billing system be the only place usage is stored, which leaves nothing to compare against when a customer asks.

One mistake is on the human side: failing to tell the customer up front how usage is counted. A short page describing the token types, how cache is charged and when the billing period closes gives customers enough to check their own logs against the invoice before they need to send an email.

If a job posting mentions usage-based billing or metering, put a specific line on the CV, such as "designed a metering pipeline with deduplication and daily reconciliation". In interviews, walk through how you traced a discrepancy down to individual events.

## This week's exercise

Build a simulated ledger of 1,000 model calls, some with cache reads. Deliberately introduce the three bugs: count only `input_tokens`, have 5% of requests retry with a random identifier, and let a few events arrive after the period cutoff. Then write the reconciliation and see whether it surfaces all three.

If the reconciliation finds all three bugs on synthetic data, you will be better prepared the next time a customer sends a CSV file along with a question.

**Try this week:**

- Pull 20 real responses from your logs, compute total input with the three-field formula and compare it with the number your metering system is sending.
- Read the code that sends usage and answer one question: if this request were sent twice, would both sends carry the same identifier?
- Write a query that compares the internal ledger with the meter summary per customer for yesterday.

## Sources

- [Record usage for billing with the API](https://docs.stripe.com/billing/subscriptions/usage-based/recording-usage-api)

- [Configure an invoice finalization grace period](https://docs.stripe.com/billing/subscriptions/usage-based/configure-grace-period)

- [Prompt caching - Claude API docs](https://platform.claude.com/docs/en/build-with-claude/prompt-caching)

- [Usage & Billing — ReqLLM](https://req-llm.hexdocs.pm/usage-and-billing.html)

- [Event ingestion architecture (Lago)](https://getlago.com/blog/event-ingestion-architecture)
