How Metergraph works

Metergraph is a queue, an HTTP endpoint, an object store, a worker, a Postgres database and a dashboard. Nothing about it is clever. What matters is where each boundary is, because that is what decides how a failure behaves and who can see what.

The path a call takes

One call, six hops. Everything after the 202 is asynchronous.

In your process. wrap() intercepts the provider method, lets the real call run untouched, and afterwards builds a normalized row: timestamps, model, provider, token counts, latency, status, the calling function, the ambient route and trace, your tags, and (unless you turned it off) the scrubbed request and response. The row goes on a bounded in-memory queue. A background thread drains it on a timer, gzips a batch, and posts it.

At the ingest API. The bearer token is resolved to a workspace, the monthly allowance is checked, credential-shaped fields are stripped again, and any row that declared a content opt-out has its content fields removed here rather than later. The batch is then written to object storage together with a ledger row, and only then does the request return 202. That ordering is the durability promise: a 202 means the batch survives a crash.

In the worker. A pointer to the batch is queued. The worker is idempotent on (batch_key, row_idx), so a duplicate wake-up is harmless. It truncates each stored content field to 100 KiB, resolves the model against the price catalog, computes the cost, and inserts the rows.

At read time. The dashboard and the read API query Postgres directly. The analysis runner is not part of this loop at all: it is a separate one-off task that exports a window of stored calls and hands them to a pinned pipeline. See how analysis works.

Why capture is asynchronous

Because the alternative is unacceptable. If sending telemetry were part of your model call, then every Metergraph outage, every slow network, every burst of retries would become an outage or a latency spike in your product. Observability that can take down the thing it observes is not worth having.

So the provider call returns to your code the moment the provider answers, and the row is handed to a queue. The cost of that choice is honest and worth knowing: telemetry can be behind, and a process that dies takes its unsent queue with it. That is why flush() and shutdown() exist, and why serverless runtimes need one of them.

Why capture is fail-open

Every failure path in the SDK is resolved in favour of your application, never in favour of the data.

• The queue drops rather than blocks. When it is full, new rows are counted as dropped and discarded. Back-pressure from a telemetry queue into a request handler would be a self-inflicted outage. Each batch reports its own dropped count in meta, so the loss is visible rather than silent.
• init() never raises. If anything in setup fails, it logs one warning, leaves the runtime uninstrumented, and returns. Your application runs.
• A bad token disables capture, quietly. A 401 or 403 from ingest marks the transport fatal for the process, logs once, and stops trying. It does not retry forever against a key that will never work.
• Normalization failures yield the provider’s own value. If Metergraph’s bookkeeping raises while reading a streamed chunk, the raw provider chunk is returned anyway. Telemetry never drops or corrupts a token of your response.
• A rejected batch is dropped, not retried. 400, 404 and 422 are payload problems and retrying cannot fix them, so that batch goes and the process carries on. 413 is different: the batch is split in half and each half retried, down to a single row. Server errors back off exponentially to a 60 second ceiling.

The consequence to internalize: the absence of a row is not evidence that a call did not happen. Metergraph is a spend and quality instrument, not a billing ledger. Reconcile against your provider invoice, not against this.

Only instrumented paths are captured

An unwrapped client, a raw HTTP request, or a provider call in a worker that never ran init() produces no row at all. It does not error. It is invisible. Audit coverage is the standing check for this.

Where the dollar figure comes from

Metergraph does not trust the provider’s cost field and does not ask the SDK to do arithmetic. Cost is derived on the server, from an effective-dated catalog.

The catalog is a set of price rows, each carrying a model, a pricing channel, a region, and a validity window. Pricing a call means resolving the provider and raw model identifier to a canonical model, then selecting the price row whose window contains that call’s own timestamp. A call made in March is priced with March’s prices, whatever the catalog says today. The worker does this once, at ingest, and stores the result alongside the row, together with the price row it used, so any figure in the dashboard can be traced back to the exact price that produced it.

Two consequences follow, and both are deliberate.

A price correction does not rewrite your history. Rows already ingested keep the cost they were given. The catalog migration that restored server-side pricing says so in as many words: copy the value, but do not reprice or otherwise rewrite rows. This is the right trade for a figure people quote in meetings and put in board decks. The alternative, recomputing every historical total from today’s catalog, would mean last month’s number changed under you because a vendor adjusted a price this morning.

An unknown model produces no cost, not a wrong one. A model the catalog cannot resolve, or one with no price row covering its timestamp, is stored with a cost status of unpriced and a machine-readable reason. It stays fully visible in the dashboard, with its tokens and latency intact and its cost cell empty. Metergraph would rather show you a gap than a confident guess. A client-reported cost, when the SDK has one, is kept beside the catalog figure as provenance and never overrides it.

The operator tooling is different

The internal mgtool read layer computes cost at read time from a local price file, which is the right design for offline analysis over a snapshot. The product does not work that way, and the two should not be confused.

The rest of the shape

• Postgres 17 backs the hosted service and the customer-local stack. The open-source server runs Postgres 16.
• Retention on the free plan is 90 days of metadata. Content follows the workspace metadata-retention period; raw batch objects have a separate, shorter lifecycle.
• Reads are content-free by default. Trace list and summary responses carry no prompts or completions. Only the single-trace detail endpoint does, and it requires a same-workspace dashboard session. An ingest key cannot read it back, and neither can an agent key.

Content and privacy What leaves your process, what is stored, and every control you have over both.

The identity model Function, route, trace, session and tags: the five labels every row carries.