FDE Times dataFDE jobs open 924▲33 Change in open roles over 7 days, like-for-like: roles from companies we started tracking this week are left outNew in 7 days 80Companies hiring 196Remote-friendly 25%Median US pay $200kTop hirer Databricks 107
中文VI

The newspaper of the Forward Deployed Engineer

Guides

Three metering bugs that turn token invoices into disputes

Two of them can be designed out of the usage event from the start; the third needs reconciliation finished before the invoice is finalized.

ShareLinkedInFacebookX

In brief

  • With Anthropic, input_tokens counts only the tokens after the last cache breakpoint. Total input requires adding the two cache fields.
  • Retries on a slow network can easily count one call twice. A deterministic identifier for each event prevents this.
  • Stripe processes meter events asynchronously, and usage that arrives after an invoice is finalized is not billed. Reconciliation must finish within the grace period.
Stacked bar showing the 8,200 input tokens of one request: 8,000 cache_read_input_tokens fill most of the bar, while input_tokens is only 200 tokens, shaded orange at the end of the bar; cache_creation_input_tokens is 0.
With Anthropic, input_tokens counts only what follows the last cache breakpoint. Total input must add all three fields. Source: Anthropic prompt caching documentation, per the example in the article.

A customer emails a CSV file. Their logs show one token count, this month’s invoice shows another, and they want to know why. At that point the engineer who built the integration is answering a question about money, and the answer has to match line by line.

No major incident is needed for this to happen. Three small flaws baked into the design are enough: undercounting because a usage field was misread, double counting because of retries, and late counting because events arrived after the billing period closed.

For an FDE deploying an AI product billed by the token, all three belong to engineering. The finance team cannot fix them.

Each one can be prevented in advance, on one condition: every model call is treated as an accounting entry, with the rigor that implies, rather than as one more line in a log.

Is input_tokens the total input?

No, at least with Anthropic. Anthropic’s prompt caching documentation states that the input_tokens field counts only the tokens after the last cache breakpoint, which excludes much of what was actually sent. Total input is cache_read_input_tokens + cache_creation_input_tokens + input_tokens.

Take an agent with a long system prompt that has already been cached. A request returns input_tokens = 200, cache_read_input_tokens = 8000, cache_creation_input_tokens = 0 and 500 output tokens. A metering system that reads only input_tokens records 200 input tokens. The real figure is 8,200.

It gets harder because providers do not count the same way. The documentation for the ReqLLM library notes that Anthropic usage typically carries the flag input_includes_cached: false, while OpenAI’s is typically true. Applying the three-field sum to a provider that already folds cache into input counts the cache twice.

Even with a correct total, a single aggregated number should not be sent. ReqLLM points out that cache reads and cache writes have separate counters because providers may price them differently, and reasoning tokens may carry their own price too. If the customer contract has several price tiers, the event has to keep each token type separate.

Sent twice, billed once

The second bug lives in the network layer. Usage goes to the billing system, the request times out, the code retries, and the first attempt had in fact succeeded. Stripe recommends idempotency keys for precisely this case: every meter event has an identifier that can be set by the caller, and if it is left blank, Stripe generates one.

The trap is in “Stripe generates one”. If each retry lets Stripe create a fresh identifier, the two sends become two distinct events. The identifier must be deterministic, meaning it can be derived from the model call itself, for example by combining the customer ID with the provider’s request id.

Lago’s write-up of its ingestion architecture defines exactly-once along the same lines: deduplication based on event_id and an idempotency_key generated on the client side. The function below does both jobs, normalizing usage and attaching the identifier:

def build_usage_event(provider, customer_id, request_id, usage, ts):
    # usage has already been mapped to common field names; original names vary by provider
    cache_read = usage.get("cache_read", 0)
    cache_write = usage.get("cache_write", 0)
    raw_input = usage["input"]

    INPUT_INCLUDES_CACHED = {"anthropic": False, "openai": True}
    if INPUT_INCLUDES_CACHED[provider]:
        uncached_input = raw_input - cache_read - cache_write
    else:
        uncached_input = raw_input

    return {
        "identifier": f"{customer_id}:{provider}:{request_id}",
        "customer_id": customer_id,
        "timestamp": ts,  # time of the model call, not time of sending
        "uncached_input": uncached_input,
        "cache_read": cache_read,
        "cache_write": cache_write,
        "output": usage["output"],
    }

Call this function twice for the same request and it returns the same identifier. That is the whole deduplication mechanism: the downstream system sees the event already exists and discards the second send.

Note that the timestamp field is taken from the time of the model call. Stripe’s documentation requires timestamps to fall within the past 35 calendar days and no more than 5 minutes in the future. A server clock running a few minutes fast, or a backfill job that runs too late, is enough to get events rejected.

At high volume, aggregate before sending

Stripe limits Meter Events to 1,000 calls per second per account, and each customer-and-meter pair is allowed only one concurrent call. An agent running many parallel tasks for the same customer will hit the second limit very quickly.

The sensible approach is to write every call to an internal ledger, then aggregate by time window before sending. The identifier must then be deterministic per window, for example customer:meter:2026-11-17T10:05, so that resending a window can never produce two events.

The cost is a decision about when to close a window. Events that arrive after a window has been sent go into a new window. Editing the old one would break the deduplication mechanism you built.

Reconciliation must finish before the invoice locks

The third bug is the hardest to spot because nothing breaks right away. Stripe processes meter events asynchronously, so the usage totals in the meter summary and on the upcoming invoice may not yet include recently sent events. Invalid events are also reported only afterwards, through error Events that you must listen for, fix and resend.

The time available for all this is limited. By default each Stripe invoice has a 1-hour grace period before finalization, adjustable up to 72 hours, during which usage can still be reported for the previous period. Usage that arrives after finalization still appears in the meter summary but is added neither to the finalized invoice nor to the next one.

That is the situation that produces a dispute: the dashboard shows one number, the invoice another. Lago also observes that late-arriving events break reconciliation. Reconciliation therefore needs to run on a fixed schedule, well before any customer asks.

The minimum schedule has four steps. First, compare the internal ledger totals with the meter summary every day, broken down by customer and by token type. Second, for each discrepancy, trace down to individual identifiers to find out whether the event was never sent, failed, or is still pending.

Third, read the error Events Stripe returns, fix the broken events and resend them with the same original identifier. Fourth, when the period ends, rerun the comparison inside the grace period and send any missing usage before the invoice locks.

Symptom in reconciliation Common cause What to do
Ledger higher than meter, consistently on cached requests Reading only input_tokens Fix normalization, send the shortfall within the period
Meter higher than ledger Retries generating new identifiers Switch to deterministic identifiers
A few events off, scattered Asynchronous errors or timestamps outside the allowed range Read error Events, fix and resend
Discrepancy concentrated at period end Sent after the invoice was finalized Extend the grace period, reconcile earlier

Mistakes FDEs commonly make

The most common is trusting that input_tokens is the total, because the name suggests it. The next is using the send time instead of the model call time, which pushes calls made just before the period cutoff into the following period. The last is letting the billing system be the only place usage is stored, which leaves nothing to compare against when a customer asks.

One mistake is on the human side: failing to tell the customer up front how usage is counted. A short page describing the token types, how cache is charged and when the billing period closes gives customers enough to check their own logs against the invoice before they need to send an email.

If a job posting mentions usage-based billing or metering, put a specific line on the CV, such as “designed a metering pipeline with deduplication and daily reconciliation”. In interviews, walk through how you traced a discrepancy down to individual events.

This week’s exercise

Build a simulated ledger of 1,000 model calls, some with cache reads. Deliberately introduce the three bugs: count only input_tokens, have 5% of requests retry with a random identifier, and let a few events arrive after the period cutoff. Then write the reconciliation and see whether it surfaces all three.

If the reconciliation finds all three bugs on synthetic data, you will be better prepared the next time a customer sends a CSV file along with a question.

Was this article useful?

Use with your AI assistantAsk Claude ↗Ask ChatGPT ↗
5 sources
Read next on the roadmap · Stage 6: MeasurementSilence from the client is not a sign the project is going wellFive small signals warn of a failing deployment weeks in advance, and a short script lets an FDE raise them before the client does.