22 September 2026 · Research · Vasiliy Radostev

Buying inference: who makes the models, who serves them, and how it impacts how much you pay

A lot is written about language model performance, one model against another. But it is rare to find a good write-up about how to choose the inference provider. The same model can cost eight times more and/or perform differently depending on the provider. Providers can run cheaper lower-precision copies of the model, serve retired models, and have their own price tag.

This article is for engineers and engineering leaders deciding where to run an LLM once the cost matters. It explains who makes models and who sells access to them, what changes from one provider to the next, when comparison shopping pays off, and when running your own hardware makes sense. If you want the short version, there is a five-step checklist near the end.

To decide what to look at, we took a large public gateway catalogue and, on 22 September 2026, pulled the price of every provider selling each model. A few figures come from other people's published work rather than from that collection, and those are marked where they appear.

This is a companion to our low-cost model benchmark, which measured ten models on production workloads. That article asked which model to use. This one asks where to get it.

Two kinds of model makers

Models are built by two kinds of organisation, and that difference is why prices vary so much.

Frontier labs train models and keep the weights. Anthropic, OpenAI and Google are the obvious three. The weights never leave their own systems, and you can only reach the model through a route the lab has licensed.

The second group publishes the weights. Meta, DeepSeek, Alibaba's Qwen team, Mistral, Google's Gemma line, NVIDIA's Nemotron line and Moonshot all release files anyone can download and run. Google sits in both groups: Gemini is proprietary, Gemma is open weight, and the commercial dynamics around the two are not comparable.

The term people reach for is open source, but that is not accurate. Most published weights carry a licence restricting some uses, and almost none arrive with the training data or the code that produced them. Open weight is the precise term: you can download and run the model, but not necessarily rebuild it or use it for any purpose you choose.

The commercial consequence is simple. A proprietary model has one supplier, even when several platforms resell access to it. An open-weight model can be hosted by anyone, so competing providers set their own prices and their own serving setups, and both vary far more.

Four kinds of model providers

Four kinds of business sell access to models, and which one you buy from tells you what to expect on price, release timing and support.

• Model labs sell direct access to their own models: the Anthropic API, the OpenAI API, the Gemini Developer API. New models and features arrive here first, usually weeks ahead of everywhere else.
• Hyperscalers are the big clouds, which have licensed proprietary models to serve inside their own platforms: AWS Bedrock, Google Vertex AI and Microsoft Azure AI Foundry. Companies choose them for an existing contract, compliance controls they have already certified, and one consolidated bill. Vertex is an unusual case: Google sells Anthropic's Claude there alongside its own Gemini.
• Specialist inference providers serve open-weight models and nothing else: DeepInfra, Together, Fireworks, Groq, Novita, Parasail, SiliconFlow, CoreWeave and a long tail behind them. They cannot offer Claude or GPT, because no licence exists that would give them those models. They compete on price, speed and throughput.
• Routers, also called gateways or aggregators, host nothing. OpenRouter and the Vercel AI Gateway give you one API and one bill for what the other three sell.

Our benchmark used Vercel for exactly this reason: ten models from eight labs, reachable through one credential.

Model selection is different across providers

Different providers can list a similar number of models and still not offer the same selection. On 22 September 2026 the OpenRouter catalogue returned 453 entries and the Vercel AI Gateway 386. They do not carry the same models.

Part of the difference is category. Vercel lists 21 image and video models, including Flux, Imagen and Veo. OpenRouter lists none, being a text and multimodal-text catalogue. For a team that needs image generation billed alongside its text models, that alone decides the question.

Within the same category, model selection varies too. For example:

Grok 4.1 Fast is in our own benchmark, published eighteen days earlier, and is now unavailable on one of the two largest routers.

A naive analysis comparing catalogues can easily be confused about which model is offered where due to naming variation. OpenRouter lists qwen/qwen3-14b where Vercel lists alibaba/qwen-3-14b, and xAI appears under x-ai on one and spacexai on the other. For production decisions, check the specific models you depend on.

Models retired by their makers often remain callable on hyperscaler routes

A model can disappear from its maker's API while remaining available through AWS, Azure or Google Cloud, because each platform follows its own retirement schedule.

Anthropic's published deprecation schedule retired Claude 3 Haiku on its own API on 20 April 2026, and Claude Opus 4.1 on 5 August 2026. We did not verify those dates independently. What we did observe directly is that on 22 September 2026 both models were still callable through OpenRouter. Each had exactly one endpoint behind it, and in both cases that endpoint was Amazon Bedrock: Claude 3 Haiku at $0.25 input and $1.25 output per million tokens, Claude Opus 4.1 at $15.00 and $75.00.

Bedrock listed Claude Opus 4.1 for more than five months beyond Anthropic's retirement date, while Azure retired a GPT-4o snapshot before OpenAI did. Check each route rather than assuming a reseller will keep a model longer.

An alternate route can therefore give you more migration time. Before relying on it, verify that it serves the same model version and behaves the same way on your workloads, because no vendor guarantees that for you.

Whether you pay the same everywhere depends on who owns the weights

For a proprietary model you pay the same rate wherever you buy it, with a few predictable premiums on top. For an open-weight model you do not.

Every licensed route charges the same list price for a proprietary model

Published per-token rates for a proprietary model are consistent wherever it is legally sold. Anthropic and AWS Bedrock list identical rates for Claude, Azure matches OpenAI, and Vertex matches the Gemini Developer API. Prompt caching rates match, and both first-party and hyperscaler routes offer asynchronous batch at half the on-demand price.

The variation that does exist follows a pattern, and you can see it in live prices. All seven paid endpoints for GPT-5.6 Luna on OpenRouter, per million tokens:

The dearest is four times the cheapest, for one model on one platform, and the endpoint column says why. OpenAI sells the same model at three service tiers, flex, standard and fast, and the 4x is the gap between the slowest and the fastest, not competition. Azure's global endpoint matches OpenAI's standard price exactly. Azure's US and EU endpoints and Bedrock's us-east-1 all charge 0.22 and 1.32, which is 1.10 times the standard rate: the documented premium for a regional endpoint, and here it is in observed prices, not just in the documentation.

These prices are not competing providers discounting the same service. They are different service tiers, priority levels and regional premiums. Decide which of those you need first, then take the lowest-priced tier that gives it to you.

For proprietary models, the larger cost differences usually come from the surrounding cloud platform rather than the token rate. Published analyses put Azure's true production cost 15 to 40 percent above the listed token price once support plans, egress, private networking, log analytics and fine-tune model hosting are accounted for, and describe Vertex in similar terms. Provisioned throughput commitments move effective unit cost substantially in both directions, depending on how accurately you forecast your own traffic. None of that appears on the price page you were comparing.

Open-weight prices vary widely between providers

When several providers serve the same weights, their prices vary widely and stay that way. Across the 153 models served by two or more distinct providers, blended at three input tokens per output token, which approximates a structured-output workload:

Half the models with genuine competition vary by at least double, a third by triple, and the widest, DeepSeek V3.2, ranged from $0.23 to $3.38 per million blended tokens.

Colour shows how much precision each provider keeps: a 4-bit endpoint is compressed most heavily and is cheapest to run, while 16-bit is the model as released. The hollow rings are the endpoints that do not say, which is most of them.

Gemma 4 31B, from our benchmark, shows what that looks like for one model. Thirteen paid endpoints, per million tokens, sorted by input price:

Eight times separates cheapest from dearest on input. DeepInfra appears three times at three prices, selling the same model at three combinations of precision and context window. Note the two dearest rows: ModelRun charges 8.3 times DeepInfra's lowest rate and serves fp4, the same reduced precision as the cheapest endpoint in the table.

Precision, context window and jurisdiction vary underneath a single model name

A model name does not fully describe what a provider is selling. Endpoints for the same model can differ in numerical precision, context window, reliability and the country they run in. In our data, price did not reliably predict precision, and 59 percent of endpoints did not state their precision at all.

• Precision. Every model is a pile of numbers, and a provider can store them roughly instead of exactly, the way a JPEG stores a photograph with less detail than the original. This is called quantization. The Gemma table shows the three settings in use: bf16 is the model as its maker released it, fp8 stores each number in eight bits, and fp4 in four. Each step down needs less memory and less hardware, so it is cheaper to serve. A quantized copy usually answers much the same, but on work near the edge of what the model can manage the difference can show up in what you get back. Where the table says unknown, the provider did not state it.
• Context window. Gemma 4 31B above is sold with a 256K context by most providers and 128K by two of them, at different prices. If your prompts are long, part of that table is not offering the product you need.
• Operations. Uptime varies a lot between endpoints, and so does the country the inference runs in. For a company under a data residency obligation, that alone can rule out the cheapest option.

None of this makes inexpensive endpoints a poor choice. It means you cannot pick the cheapest from the price column alone, and a router set to default to the lowest price is silently making a quality decision for you.

When comparison shopping is worth the effort

Shopping for the best price costs time researching providers and extra integration work. For proprietary models from the big labs (OpenAI, Anthropic and Google) it is usually not worth it, since pricing is consistent across routes. For open-weight models on a non-trivial workload it is, because half of them differ by 2x or more between providers.

The order to make these decisions in

You need to consider which model, from what provider, and which exact endpoint. These decisions constrain one another, so the order matters.

1. Choose the model, preferably by benchmarking your own workloads rather than public leaderboards. The difference in price per task can be 10x.
2. Decide which providers and endpoints you cannot use. Consider data residency, procurement, and other constraints you might have.
3. Check which endpoints serve your needs for context size, precision and region.
4. Compare price per task for the models and endpoints that meet all of your requirements. This is where metergraph.dev can help.
5. Test regularly. Two of the ten models we tested less than a month ago are now gone from one major router, and such changes are common.

Running inference yourself makes sense if you can keep the hardware busy

Self-hosting pays only when you can keep the GPU (the key bottleneck for inference) busy. A rented GPU costs the same whether it is answering requests or sitting idle, while a hosted API charges only for what you use.

How busy is busy enough depends on which model you're using. One published estimate puts the break-even for a single card at roughly 15 million tokens a day against frontier pricing and 50 million against a mid-tier open-weight endpoint, and finds that against the cheapest open-weight endpoints a single card never comes out ahead. In our benchmark we found that a single rented L40S or A100 needed 300,000 to 2 million requests a week to break even against Gemma 4 31B on a gateway, without accounting for engineering support.

If your traffic has a lot of spikes, it is rarely worth renting your own GPUs. Self-host when you need data residency, predictable latency, vendor independence, or you have steady traffic that keeps the hardware busy. Otherwise a hosted provider is usually cheaper.

Key takeaways

Proprietary and open-weight models reach you through different routes, and only one of those routes has competing sellers. Nearly every price difference in this article comes back to that.

• Proprietary models have one supplier, so the same service tier costs the same on every licensed route. What still varies is the regional premium, about ten percent, and the platform charges around the tokens, which published analyses put 15 to 40 percent above the token rate.
• Open-weight models have competing providers, and prices differ a lot. Of the 153 models we measured with more than one provider, half vary by 2x or more and the widest by 14.42x.
• The cheapest endpoint may differ in precision, context limit or operating region. 59 percent of the 1,317 endpoints we collected do not state their precision, and price is not a reliable guide to it.
• Availability changes. Two models from our September benchmark left a major router within eighteen days, while models retired by their makers stayed available on Bedrock for months. Check the pricing of the endpoints you depend on regularly.
• Self-hosting makes financial sense only with steady traffic that keeps the hardware busy. Choose it for residency, latency or vendor independence; if your traffic has a lot of spikes, a hosted provider is usually cheaper.

Method and limitations

Catalogue and endpoint data were collected on 22 September 2026 from the OpenRouter public API, covering every paid model and every provider endpoint behind it, and from the Vercel AI Gateway model listing. Collection was a single pass with no failures, yielding 406 paid models, 1,317 paid endpoints and 153 models served by more than one provider. Free endpoints were excluded, since a zero price distorts a ratio.

We assumed a synthetic summarization workload, with three input tokens per output token. Different workloads will result in a different blend and a different order of providers.

Spreads are computed across endpoints on a single router. A model may be cheaper from a provider that does not list there, so these are lower bounds on true market dispersion.

Precision is taken from each provider's own declaration. Because 59 percent declare nothing, the quantization analysis covers a minority of the catalogue and may not generalise to the rest.

We did not verify that a model served by two providers produces equivalent output.

Proprietary pricing parity across first-party and hyperscaler routes is drawn from published rate cards and secondary analysis, corroborated against the live endpoint data above. First-party retirement dates, and the comparison of schedules across platforms, come from vendor lifecycle documentation and a third-party compilation of it. We verified the resulting availability ourselves but not the dates behind it. The 15 to 40 percent platform-cost figure for Azure and the self-hosting break-even figures come from third-party analyses we have not reproduced, and the text says so where they appear.

References

• OpenRouter models and endpoints API: every price, precision and availability figure.
• Vercel AI Gateway model list: the second catalogue.
• Amazon Bedrock pricing and lifecycle: Bedrock prices and retirement dates for the retired Claude models.
• Microsoft Foundry model retirements: Azure's schedule, including the GPT-4o case.
• Anthropic pricing and deprecations: first-party Claude prices and retirement dates.
• bex.co, July 2026: the self-hosting break-even estimates.

Catalogue and price data collected 22 September 2026 from the OpenRouter and Vercel AI Gateway public APIs. Prices move weekly; figures should be re-verified before you act on them.

Questions or suggestions: vasiliy@psl.com