Skip to main content
DanManREAL/ AI_LAB
Updated Nov 20, 2026Request a test

Public benchmarkExternal source reviewReviewed by Published Source checked Updated

Sol 6.2: new model and public coding benchmark evidence

The report

Public benchmark

Verdict

Published benchmark evidence

Sol 6.2's 43.8% score sits only 0.4 points below Astra's listed 44.2% on Cognition FrontierCode 2.0, at $1.29 rather than $5.97 reported mean rollout cost.

On the tier list: The November 17 tier is Dan’s own ranking; public benchmark numbers are a separate evidence channel, not an automatic tier adjustment.

What the benchmarks report

  • FrontierCode 2.0 — Cognition: 43.8%. Sol 6.2; max; Codex harness. Reported by Cognition benchmark operator.
  • FrontierCode 2.0 mean rollout cost: $1.29. Sol 6.2; max; benchmark API-price estimate. Reported by Cognition benchmark operator.
  • FrontierCode 2.0 gap to Astra 6.2: within 0.5 percentage points. Sol 6.2 and Astra 6.2 at max; provider summary. Reported by OpenAI vendor release.

Original sources

Source 1 — FrontierCode 2.0 — Cognition, FrontierCode 2.0 mean rollout cost.

Source 2 — FrontierCode 2.0 gap to Astra 6.2.

What this evidence tells a reader

For teams testing a less expensive coding agent, Sol 6.2 has a concrete public comparison point: Cognition's FrontierCode 2.0 table lists both Sol and Astra at max effort in a Codex harness. The 43.8% versus 44.2% figures are close enough that a reader should ask about run-to-run uncertainty before declaring either model more capable.

The cost figures are more visibly separated. Cognition's $1.29 mean rollout for Sol versus $5.97 for Astra is about a 78% reduction in the quoted benchmark mean. That does not mean every software task is 78% cheaper: token usage, retries, pricing agreements and human review costs are not captured by a single mean.

If a workflow has many short, reviewed coding attempts, Sol may be worth trialing first on cost grounds. If errors create substantial recovery work, cost per accepted patch—not cost per raw rollout—becomes the more useful measure.

How to use these results

  • Use the same repository issue and acceptance criteria when trialing Sol against a larger model.
  • Record retries and human repair time; a cheap unsuccessful run is not necessarily economical.
  • Do not convert the provider's percentage-point description into a universal reliability guarantee.
The comparison is a reader-facing interpretation of Cognition's listed results, not a head-to-head run performed by DanManREAL.

Limits of this report

  • The Cognition listing supplies mean rollout costs at public API prices, not the site's real bill or a subscription comparison.
  • The near-equal scores do not establish equal outcomes across larger repositories, different harnesses, or untested reasoning efforts.

Follow the lab

New test reports and tier-list changes get announced in my Discord's #news channel. The full videos are on YouTube.

Receipts

4 items

DanManREAL · AI lab

Test receipt

Public benchmarkNov 17, 2026

GPT-6.2 Sol

FrontierCode 2.0 — Cognition
43.8%Cognition benchmark operator (independent operator or secondary publication). Sol 6.2; max; Codex harness. Not independently rerun by DanManREAL.
FrontierCode 2.0 mean rollout cost
$1.29Cognition benchmark operator (independent operator or secondary publication). Sol 6.2; max; benchmark API-price estimate. Not independently rerun by DanManREAL.
FrontierCode 2.0 gap to Astra 6.2
within 0.5 percentage pointsOpenAI vendor release (model publisher). Sol 6.2 and Astra 6.2 at max; provider summary. Not independently rerun by DanManREAL.
Source review
November 17, 2026Date these source claims were reviewed; not a claim about when the published benchmark was executed.

Verdict

Published benchmark evidence

Tier list The November 17 tier is Dan’s own ranking; public benchmark numbers are a separate evidence channel, not an automatic tier adjustment.

danmanreal.com/tests/gpt-6-2-sol-public-evidence

Models covered by external benchmarks

GPT-6.2 Sol

On the tier list's bench

Related

Independent testing. Sponsorships are always disclosed, and verdicts are never for sale.

Disagree? Bring your receipts

All reports

36 reports