Public benchmarkExternal source reviewReviewed by DanPublished Source checked Updated
Sol 6.2: new model and public coding benchmark evidence
The report
Verdict
Published benchmark evidence
Sol 6.2's 43.8% score sits only 0.4 points below Astra's listed 44.2% on Cognition FrontierCode 2.0, at $1.29 rather than $5.97 reported mean rollout cost.
On the tier list: The November 17 tier is Dan’s own ranking; public benchmark numbers are a separate evidence channel, not an automatic tier adjustment.
What the benchmarks report
- FrontierCode 2.0 — Cognition: 43.8%. Sol 6.2; max; Codex harness. Reported by Cognition benchmark operator.
- FrontierCode 2.0 mean rollout cost: $1.29. Sol 6.2; max; benchmark API-price estimate. Reported by Cognition benchmark operator.
- FrontierCode 2.0 gap to Astra 6.2: within 0.5 percentage points. Sol 6.2 and Astra 6.2 at max; provider summary. Reported by OpenAI vendor release.
Original sources
Source 1 — FrontierCode 2.0 — Cognition, FrontierCode 2.0 mean rollout cost.
Source 2 — FrontierCode 2.0 gap to Astra 6.2.
What this evidence tells a reader
For teams testing a less expensive coding agent, Sol 6.2 has a concrete public comparison point: Cognition's FrontierCode 2.0 table lists both Sol and Astra at max effort in a Codex harness. The 43.8% versus 44.2% figures are close enough that a reader should ask about run-to-run uncertainty before declaring either model more capable.
The cost figures are more visibly separated. Cognition's $1.29 mean rollout for Sol versus $5.97 for Astra is about a 78% reduction in the quoted benchmark mean. That does not mean every software task is 78% cheaper: token usage, retries, pricing agreements and human review costs are not captured by a single mean.
If a workflow has many short, reviewed coding attempts, Sol may be worth trialing first on cost grounds. If errors create substantial recovery work, cost per accepted patch—not cost per raw rollout—becomes the more useful measure.
How to use these results
- Use the same repository issue and acceptance criteria when trialing Sol against a larger model.
- Record retries and human repair time; a cheap unsuccessful run is not necessarily economical.
- Do not convert the provider's percentage-point description into a universal reliability guarantee.
Limits of this report
- The Cognition listing supplies mean rollout costs at public API prices, not the site's real bill or a subscription comparison.
- The near-equal scores do not establish equal outcomes across larger repositories, different harnesses, or untested reasoning efforts.
Follow the lab
New test reports and tier-list changes get announced in my Discord's #news channel. The full videos are on YouTube.
Receipts
DanManREAL · AI lab
Test receipt
GPT-6.2 Sol
- FrontierCode 2.0 — Cognition
- 43.8%Cognition benchmark operator (independent operator or secondary publication). Sol 6.2; max; Codex harness. Not independently rerun by DanManREAL.
- FrontierCode 2.0 mean rollout cost
- $1.29Cognition benchmark operator (independent operator or secondary publication). Sol 6.2; max; benchmark API-price estimate. Not independently rerun by DanManREAL.
- FrontierCode 2.0 gap to Astra 6.2
- within 0.5 percentage pointsOpenAI vendor release (model publisher). Sol 6.2 and Astra 6.2 at max; provider summary. Not independently rerun by DanManREAL.
- Source review
- November 17, 2026Date these source claims were reviewed; not a claim about when the published benchmark was executed.
Verdict
Published benchmark evidence
Tier list The November 17 tier is Dan’s own ranking; public benchmark numbers are a separate evidence channel, not an automatic tier adjustment.
danmanreal.com/tests/gpt-6-2-sol-public-evidence
Models covered by external benchmarks
On the tier list's bench
- GPT-6.2 Sol · 1 receipt on file
Related
Independent testing. Sponsorships are always disclosed, and verdicts are never for sale.
Disagree? Bring your receiptsAll reports
- One inbox for my AI coding agents: triage took 34% less time, resuming work did not improveHead-to-headNov 20
- Mythos 5.2: research and security benchmark source reviewPublic benchmarkNov 17
- Fable 5.2: agentic coding benchmark source reviewPublic benchmarkNov 17
- Claude Opus 5.5: public coding benchmark receiptsPublic benchmarkNov 17
- Astra 6.2: new model and public task-evaluation evidencePublic benchmarkNov 17
- Sol 6.2: new model and public coding benchmark evidencePublic benchmarkNov 17
- Gemini 4 Carbon: benchmark source and usage profilePublic benchmarkNov 17
- Qwen 4: Qwen4-Max launch and benchmark sourcesPublic benchmarkNov 17
- Kimi K3.1 Pro: public coding benchmark receiptsPublic benchmarkNov 17
- GLM-5.3: public coding benchmark receiptsPublic benchmarkNov 17
- Gemini 4 Argon: public coding benchmark receiptsPublic benchmarkNov 17
- Claude Sonnet 5.5: public coding benchmark receiptsPublic benchmarkNov 17
- DeepSeek V4.1 Flash: public coding benchmark receiptsPublic benchmarkNov 17
- Muse Wonder Full 1.0: public coding benchmark receiptsPublic benchmarkNov 17
- Grok 4.7: separate release and public coding resultsPublic benchmarkNov 17
- ai-design-check on two of my FABLE screens: 10 warnings, 4 of them false alarmsField testNov 2
- Vercel Agent Desktop used a sample Trip app in 6 actions without moving the mouseField testNov 1
- Cerberus Guard on sample data: one crawler name, two verdicts, and a block earned by behaviorField testOct 31
- GPT-6.1 Terra vs Opus 5.5: three real projects, and the designs I picked blindHead-to-headOct 29
- Steps → Reduced: opening a terminal agent went from 98 seconds to 11AuditOct 28
- One ChatGPT chat directed Opus and Codex at once through DMR_REMOTEField testOct 28
- Claude Duet ran my channel for a week and caught 269 fake AI crawlersField testOct 27
- Kimi K3.1 Pro built my animation family, after a quota stop and 21 minutes of failed requestsField testOct 24
- ChatGPT + Codex built 33 new DMR_REMOTE toolsField testOct 23
- Gemini 4 Argon reviewed a finished episode: 14 clips, one flag, one inconsistencyField testOct 21
- Qwen 3.8 Flash Next on my RTX 5080: 81 tokens a second with 202K tokens readAuditOct 21
- Does an extra AI review step pay off? A small DeepSeek trial said noHead-to-headOct 19
- My real DeepSeek bill: 12.31 billion tokens for $122.28Cost receiptOct 17
- Gemini 4 Argon kept hanging in Gemini CLI. The cause was a 60-second timeoutIncidentOct 14
- I asked 10 AI coding agents for their biggest weaknessSelf-reportOct 14
- Jarvis 27B on my own PC: 30 tokens a second, and it still recalls at 251KAuditOct 10
- My agents' first game benchmark was wrong. Here's how it was caughtIncidentOct 10
- Pixal3D + Blender turned my 2D art into animated 3D charactersField testSep 22
- DeepSeek Flash answered 200 OK and sent nothing for two hoursIncidentSep 14
- Grok, Codex and DeepSeek played WorldOS for Rank 1. Grok won with 98Head-to-headAug 20
- DeepSeek V4 Flash audited 18 of my real coding sessions. Two said 'done' too earlyAuditAug 4