Three cats representing GPT Sol, Gemini and Qwen on an astronomer’s cabin porch in the Wild West.

New models tested: Gemini 4 Hydrogen, Qwen4-Max, and GPT-6.2

Summary

I tested Gemini 4 Hydrogen, Qwen4-Max, and GPT-6.2 Astra and Sol across my standard five workloads.

Can you guess whether Hydrogen is above or below Argon? I expected a faster version of Argon based on the name, but it was actually smarter, slower, and more expensive. Google is doing a great job of confusing us.

GPT-6.2 Sol is a faster version of GPT-6.1 Sol. GPT-6.2 Astra led on coding, but like the new Fable, it comes at a steep price.

Qwen4-Max is still behind the latest Sol, but it narrowed the gap. It is competitive on quality and cost, though it is still significantly slower.

GPT-6.2 Sol and Astra

GPT-6.2 Sol was nearly as strong as GPT-6.1 Sol overall and substantially faster. Astra did best on coding among the models in this run, but fell behind on extraction and cost several times more per task. Scores vary a little across workloads, but the two Sol models show similar strengths overall.

ModelMean qualityCost / taskLatency
GPT-6.2 Sol0.913$0.034216.4 s
GPT-6.1 Sol0.919$0.033925.3 s
GPT-6.2 Astra0.904$0.127822.9 s
GPT-6 Astra0.900$0.171928.0 s
Quality by workload
ModelDesignCodingExtractionMatchingResearch
GPT-6.2 Sol0.7260.9050.9730.9631.000
GPT-6.1 Sol0.7460.9050.9910.9820.971
GPT-6.2 Astra0.7190.9280.9110.9601.000
GPT-6 Astra0.6430.9070.9910.9571.000

My takeaway: Start with GPT-6.2 Sol; consider Astra where coding quality matters enough to justify the cost.

Gemini 4 Hydrogen and Argon

Hydrogen scored higher than Argon overall in this run, with stronger design and research results, but it was slower and more expensive.

ModelMean qualityCost / taskLatency
Gemini 4 Hydrogen0.859$0.077853.8 s
Gemini 4 Argon0.844$0.043040.7 s
Gemini 3.8 Flash0.762$0.043953.5 s
Quality by workload
ModelDesignCodingExtractionMatchingResearch
Gemini 4 Hydrogen0.5430.8110.9770.9680.998
Gemini 4 Argon0.4530.8250.9960.9910.953
Gemini 3.8 Flash0.1670.7020.9950.9820.963

My takeaway: Use Hydrogen for design tasks, but otherwise I would prefer Argon.

Qwen4-Max

Qwen4-Max scored higher overall than Qwen 3.8 Max. Its workload scores were close to GPT-6 Sol’s, but it was much slower, particularly on coding.

ModelMean qualityCost / taskLatency
Qwen4-Max0.877$0.0428102.5 s
Qwen 3.8 Max0.830$0.093374.3 s
GPT-6 Sol0.881$0.029112.2 s
GPT-6.2 Sol0.913$0.034216.4 s
Quality by workload
ModelDesignCodingExtractionMatchingResearch
Qwen4-Max0.6230.8590.9610.9760.965
Qwen 3.8 Max0.5030.7420.9360.9691.000
GPT-6 Sol0.6480.8770.9450.9650.971
GPT-6.2 Sol0.7260.9050.9730.9631.000

My takeaway: A capable model, but coding tasks took about five minutes on average. It is unfortunate that it is not open-weight anymore, which was its main selling point in the past.

Testing results

Higher quality is better. The charts include only results meeting the coverage threshold.

Design quality

GPT-6.1 Sol
0.746
GPT-6.2 Sol
0.726
GPT-6.2 Astra
0.719
Qwen4-Max
0.623
Gemini 4 Hydrogen
0.543
Gemini 4 Argon
0.453

Cost per task

Five-workload mean; lower is better.

GPT-6.1 Sol
$0.0339
GPT-6.2 Sol
$0.0342
Qwen4-Max
$0.0428
Gemini 4 Argon
$0.0430
Gemini 4 Hydrogen
$0.0778
GPT-6.2 Astra
$0.1278

Response time

Five-workload mean in seconds; lower is better.

GPT-6.2 Sol
16.4 s
GPT-6.2 Astra
22.9 s
GPT-6.1 Sol
25.3 s
Gemini 4 Argon
40.7 s
Gemini 4 Hydrogen
53.8 s
Qwen4-Max
102.5 s

Full benchmark data

Each row is a model and a workload. Quality is on a 0–1 scale. Costs and latency are means of successfully measured requests for that workload, excluding judge and setup costs. Under-covered scores are shown for inspection, not ranked.

Download the full results JSON · Download results as CSV

ModelWorkloadQualityCost / taskLatency
GPT-6.1 SolDesign0.746$0.063074.5 s
GPT-6.1 SolCoding0.905$0.079032.0 s
GPT-6.1 SolExtraction0.991$0.01107.7 s
GPT-6.1 SolMatching0.982$0.00222.4 s
GPT-6.1 SolResearch0.971$0.01459.8 s
GPT-6.2 SolDesign0.726$0.058928.1 s
GPT-6.2 SolCoding0.905$0.086540.8 s
GPT-6.2 SolExtraction0.973$0.01118.1 s
GPT-6.2 SolMatching0.963$0.00211.5 s
GPT-6.2 SolResearch1.000$0.01233.8 s
GPT-6.2 AstraDesign0.719$0.220955.5 s
GPT-6.2 AstraCoding0.928$0.302434.2 s
GPT-6.2 AstraExtraction0.911$0.057212.4 s
GPT-6.2 AstraMatching0.960$0.00903.5 s
GPT-6.2 AstraResearch1.000$0.04978.9 s
Gemini 4 ArgonDesign0.453$0.080379.7 s
Gemini 4 ArgonCoding0.825$0.105675.7 s
Gemini 4 ArgonExtraction0.996$0.013020.9 s
Gemini 4 ArgonMatching0.991$0.00229.2 s
Gemini 4 ArgonResearch0.953$0.013817.9 s
Gemini 4 HydrogenDesign0.543$0.1466104.8 s
Gemini 4 HydrogenCoding0.811$0.1942131.1 s
Gemini 4 HydrogenExtraction0.977$0.022316.6 s
Gemini 4 HydrogenMatching0.968$0.00675.5 s
Gemini 4 HydrogenResearch0.998$0.019110.9 s
Qwen4-MaxDesign0.623$0.0565146.9 s
Qwen4-MaxCoding0.859$0.1295302.3 s
Qwen4-MaxExtraction0.961$0.011929.8 s
Qwen4-MaxMatching0.976$0.00215.1 s
Qwen4-MaxResearch0.965$0.014028.4 s

Methodology

Five workloads, eight cases per workload, two repetitions. Sonnet 5.5 graded saved responses using the published criteria and case-specific design rubrics. Quality scores reflect rubric performance, not task pass rates. Overall quality, cost and latency give equal weight to each workload; this can differ from averaging all successful requests together.

Incomplete outputs and explicit format failures score zero; provider failures are excluded. A model/workload result needs at least 80% coverage to be included in comparisons. Sampling and judge variation can affect comparisons.

Benchmark on your own workloads

Metergraph reads the traces your AI agents already produce, finds where you overpay or lose quality, and tests model changes on your own traffic before you switch.