See what every LLM call costs. Prove a cheaper one holds quality. Then ship it.

One wrap, both languages, no framework to adopt.

The SDK wraps the provider client you already constructed. Your call sites do not change, your types do not change, and removing it is deleting one line.

# the client you already have
from openai import OpenAI
import metergraph

client = metergraph.wrap(OpenAI())

# the stack walk names the function; route() groups it
@metergraph.track
def classify(ticket):
    with metergraph.route("ticket-classifier"):
        return client.chat.completions.create(...)

// the client you already have
import OpenAI from "openai";
import * as mg from "metergraph";

const client = mg.wrap(new OpenAI());

// name first, so a bundler cannot rename it
export const classify = mg.track(
  "billing.classify",
  () => mg.route("ticket-classifier",
    () => client.chat.completions.create(...)));

// the client you already have
import OpenAI from "openai";
import * as mg from "metergraph";

const client = mg.wrap(new OpenAI());

// name first, so a bundler cannot rename it
export const classify = mg.track(
  "billing.classify",
  () => mg.route("ticket-classifier",
    () => client.chat.completions.create(...)));

What the SDK sends, and where it stops.

The self-hosted server and the hosted service answer this differently. Both answers are below, because averaging them would be useless to anyone doing a security review.

Change one environment variable and tracking becomes a standing loop.

Same SDK, same routes, same call sites. The hosted engine picks up where the dashboard stops.

1. Step 01

   Capture

   The SDK or native OpenTelemetry GenAI ingestion logs every call and fingerprints recurring call sites into routes.

2. Step 02

   Watch

   Per-route anomaly detection from week one: spend spikes, retry loops on billable errors, cost-per-unit drift. Alerts arrive with a diagnosis rather than a red line on a chart.

3. Step 03

   Shadow

   Sampled production inputs get replayed against challenger models and configurations on your own accounts.1 Never in your request path.

4. Step 04

   Score

   Per-characteristic non-inferiority testing with bootstrap confidence intervals, plus cost and latency deltas. Everything is denominated in your unit economics: cost per answer, per resolution, per report.

5. Step 05

   Act

   Scorecard to pull request to session-sticky canary to promotion, with rollback on a config flag. Eval sets get committed to your repo as code.

• Replay runs inside your allow-lists, residency rules and unit-cost ceilings. A challenger you have not allow-listed never gets called at all. ↩

Model swaps are the headline. The rest of the list is where the hours are.

You already know most of this is worth doing. Each row is its own project, which is the whole reason it stays in the backlog.

Schedule A · Levers, typical yield, and the work each one actually is

Findings arrive looking like this

• support-triage 4.2M calls/mo · $0.019/answer · parity 97±2% −$38k/yr
• contract-extract batch-eligible · deferred 4 months −30%, PR ready
• report-writer web search flipped to full-token pricing now 41% of cost per answer
• rag-answers retry loop billing on failed calls $1.1k/hr, caught in 4 min

Free to track. Hosted to act.

Same SDK and the same routes either way. What changes is whether Metergraph shows you the cost or goes and does something about it.

Schedule B · What each edition includes

What engineers ask before they install it.