
New models tested: Gemini 4 Hydrogen, Qwen4-Max, and GPT-6.2
Summary
I tested Gemini 4 Hydrogen, Qwen4-Max, and GPT-6.2 Astra and Sol across my standard five workloads.
Can you guess whether Hydrogen is above or below Argon? I expected a faster version of Argon based on the name, but it was actually smarter, slower, and more expensive. Google is doing a great job of confusing us.
GPT-6.2 Sol is a faster version of GPT-6.1 Sol. GPT-6.2 Astra led on coding, but like the new Fable, it comes at a steep price.
Qwen4-Max is still behind the latest Sol, but it narrowed the gap. It is competitive on quality and cost, though it is still significantly slower.
GPT-6.2 Sol and Astra
GPT-6.2 Sol was nearly as strong as GPT-6.1 Sol overall and substantially faster. Astra did best on coding among the models in this run, but fell behind on extraction and cost several times more per task. Scores vary a little across workloads, but the two Sol models show similar strengths overall.
| Model | Mean quality | Cost / task | Latency |
|---|---|---|---|
| GPT-6.2 Sol | 0.913 | $0.0342 | 16.4 s |
| GPT-6.1 Sol | 0.919 | $0.0339 | 25.3 s |
| GPT-6.2 Astra | 0.904 | $0.1278 | 22.9 s |
| GPT-6 Astra | 0.900 | $0.1719 | 28.0 s |
Quality by workload
| Model | Design | Coding | Extraction | Matching | Research |
|---|---|---|---|---|---|
| GPT-6.2 Sol | 0.726 | 0.905 | 0.973 | 0.963 | 1.000 |
| GPT-6.1 Sol | 0.746 | 0.905 | 0.991 | 0.982 | 0.971 |
| GPT-6.2 Astra | 0.719 | 0.928 | 0.911 | 0.960 | 1.000 |
| GPT-6 Astra | 0.643 | 0.907 | 0.991 | 0.957 | 1.000 |
My takeaway: Start with GPT-6.2 Sol; consider Astra where coding quality matters enough to justify the cost.
Gemini 4 Hydrogen and Argon
Hydrogen scored higher than Argon overall in this run, with stronger design and research results, but it was slower and more expensive.
| Model | Mean quality | Cost / task | Latency |
|---|---|---|---|
| Gemini 4 Hydrogen | 0.859 | $0.0778 | 53.8 s |
| Gemini 4 Argon | 0.844 | $0.0430 | 40.7 s |
| Gemini 3.8 Flash | 0.762 | $0.0439 | 53.5 s |
Quality by workload
| Model | Design | Coding | Extraction | Matching | Research |
|---|---|---|---|---|---|
| Gemini 4 Hydrogen | 0.543 | 0.811 | 0.977 | 0.968 | 0.998 |
| Gemini 4 Argon | 0.453 | 0.825 | 0.996 | 0.991 | 0.953 |
| Gemini 3.8 Flash | 0.167 | 0.702 | 0.995 | 0.982 | 0.963 |
My takeaway: Use Hydrogen for design tasks, but otherwise I would prefer Argon.
Qwen4-Max
Qwen4-Max scored higher overall than Qwen 3.8 Max. Its workload scores were close to GPT-6 Sol’s, but it was much slower, particularly on coding.
| Model | Mean quality | Cost / task | Latency |
|---|---|---|---|
| Qwen4-Max | 0.877 | $0.0428 | 102.5 s |
| Qwen 3.8 Max | 0.830 | $0.0933 | 74.3 s |
| GPT-6 Sol | 0.881 | $0.0291 | 12.2 s |
| GPT-6.2 Sol | 0.913 | $0.0342 | 16.4 s |
Quality by workload
| Model | Design | Coding | Extraction | Matching | Research |
|---|---|---|---|---|---|
| Qwen4-Max | 0.623 | 0.859 | 0.961 | 0.976 | 0.965 |
| Qwen 3.8 Max | 0.503 | 0.742 | 0.936 | 0.969 | 1.000 |
| GPT-6 Sol | 0.648 | 0.877 | 0.945 | 0.965 | 0.971 |
| GPT-6.2 Sol | 0.726 | 0.905 | 0.973 | 0.963 | 1.000 |
My takeaway: A capable model, but coding tasks took about five minutes on average. It is unfortunate that it is not open-weight anymore, which was its main selling point in the past.
Testing results
Higher quality is better. The charts include only results meeting the coverage threshold.
Design quality
Cost per task
Five-workload mean; lower is better.
Response time
Five-workload mean in seconds; lower is better.
Full benchmark data
Each row is a model and a workload. Quality is on a 0–1 scale. Costs and latency are means of successfully measured requests for that workload, excluding judge and setup costs. Under-covered scores are shown for inspection, not ranked.
Download the full results JSON · Download results as CSV
| Model | Workload | Quality | Cost / task | Latency |
|---|---|---|---|---|
| GPT-6.1 Sol | Design | 0.746 | $0.0630 | 74.5 s |
| GPT-6.1 Sol | Coding | 0.905 | $0.0790 | 32.0 s |
| GPT-6.1 Sol | Extraction | 0.991 | $0.0110 | 7.7 s |
| GPT-6.1 Sol | Matching | 0.982 | $0.0022 | 2.4 s |
| GPT-6.1 Sol | Research | 0.971 | $0.0145 | 9.8 s |
| GPT-6.2 Sol | Design | 0.726 | $0.0589 | 28.1 s |
| GPT-6.2 Sol | Coding | 0.905 | $0.0865 | 40.8 s |
| GPT-6.2 Sol | Extraction | 0.973 | $0.0111 | 8.1 s |
| GPT-6.2 Sol | Matching | 0.963 | $0.0021 | 1.5 s |
| GPT-6.2 Sol | Research | 1.000 | $0.0123 | 3.8 s |
| GPT-6.2 Astra | Design | 0.719 | $0.2209 | 55.5 s |
| GPT-6.2 Astra | Coding | 0.928 | $0.3024 | 34.2 s |
| GPT-6.2 Astra | Extraction | 0.911 | $0.0572 | 12.4 s |
| GPT-6.2 Astra | Matching | 0.960 | $0.0090 | 3.5 s |
| GPT-6.2 Astra | Research | 1.000 | $0.0497 | 8.9 s |
| Gemini 4 Argon | Design | 0.453 | $0.0803 | 79.7 s |
| Gemini 4 Argon | Coding | 0.825 | $0.1056 | 75.7 s |
| Gemini 4 Argon | Extraction | 0.996 | $0.0130 | 20.9 s |
| Gemini 4 Argon | Matching | 0.991 | $0.0022 | 9.2 s |
| Gemini 4 Argon | Research | 0.953 | $0.0138 | 17.9 s |
| Gemini 4 Hydrogen | Design | 0.543 | $0.1466 | 104.8 s |
| Gemini 4 Hydrogen | Coding | 0.811 | $0.1942 | 131.1 s |
| Gemini 4 Hydrogen | Extraction | 0.977 | $0.0223 | 16.6 s |
| Gemini 4 Hydrogen | Matching | 0.968 | $0.0067 | 5.5 s |
| Gemini 4 Hydrogen | Research | 0.998 | $0.0191 | 10.9 s |
| Qwen4-Max | Design | 0.623 | $0.0565 | 146.9 s |
| Qwen4-Max | Coding | 0.859 | $0.1295 | 302.3 s |
| Qwen4-Max | Extraction | 0.961 | $0.0119 | 29.8 s |
| Qwen4-Max | Matching | 0.976 | $0.0021 | 5.1 s |
| Qwen4-Max | Research | 0.965 | $0.0140 | 28.4 s |
Methodology
Five workloads, eight cases per workload, two repetitions. Sonnet 5.5 graded saved responses using the published criteria and case-specific design rubrics. Quality scores reflect rubric performance, not task pass rates. Overall quality, cost and latency give equal weight to each workload; this can differ from averaging all successful requests together.
Incomplete outputs and explicit format failures score zero; provider failures are excluded. A model/workload result needs at least 80% coverage to be included in comparisons. Sampling and judge variation can affect comparisons.