Head-to-headTested AIBy DanPublished Tested Updated
Does an extra AI review step pay off? A small DeepSeek trial said no
All reports
- ChatGPT + Codex built 33 new DMR_REMOTE toolsField testOct 23
- Gemini 4 Argon reviewed a finished episode: 14 clips, one flag, one inconsistencyField testOct 21
- Does an extra AI review step pay off? A small DeepSeek trial said noHead-to-headOct 19
- My real DeepSeek bill: 12.31 billion tokens for $122.28Cost receiptOct 17
- Gemini 4 Argon kept hanging in Gemini CLI. The cause was a 60-second timeoutIncidentOct 14
- I asked 8 AI coding agents for their biggest weaknessSelf-reportOct 14
- My agents' first game benchmark was wrong. Here's how it was caughtIncidentOct 10
- DeepSeek Flash answered 200 OK and sent nothing for two hoursIncidentSep 14
- DeepSeek V4 Flash audited 18 of my real coding sessions. Two said 'done' too earlyAuditAug 4
The report
Verdict
Extra review cost more on both repairs
On two real repair tasks, a mandatory independent review made the work slower and more expensive with the same final result. The trial is too small to generalise from, and its formal statistical comparison was declared invalid.
On the tier list: Two workflows on the same model, so it ranks no agent against another.
| Task | Direct | Extra review | Result |
|---|---|---|---|
| Full-capacity move | 182.43 s · $0.01541 | 313.91 s · $0.03902 | Both pass |
| Shrink then widen | 175.14 s · $0.01698 | 188.06 s · $0.02157 | Both pass |
The question
A common habit with AI coding agents is to make a second agent review every change before it counts. That costs time and money. I wanted to know whether it buys anything on the small repairs my agents do every day.
How it ran
Two real bugs were given to the same DeepSeek API route. Each was fixed twice: once directly, and once with a required review by a second agent. Each attempt had up to eight turns. The same tests, hidden from the agents, checked the final changes. The trial stopped after both pairs at a checkpoint planned in advance.
What happened
Both arms fixed both defects. The review arm took longer and cost more on both tasks: about 72% more time and 2.5 times the cost on the first, and a smaller gap on the second.
Why the statistics do not count
The trial was built to end in a formal statistical comparison. That comparison was declared invalid, because the test controller logged its metrics incorrectly. The output cap was not enforced either: 56,791 output tokens were observed. The timings and costs above describe these two repairs. The broken statistical comparison cannot support a broader conclusion.
Limits of this report
- Two tasks on known, selected defects: far too few to generalise from.
- Costs are token-price estimates, not reconciled billing.
- The request used the deepseek-flash alias; which checkpoint served it was not verified.
- Times are active pipeline seconds, not end-to-end wall time; supervisor and human time were not measured.
Receipts
DanManREAL · AI lab
Test receipt
DeepSeek V4.1 Flash
- Task 1 time: direct vs extra review
- 182.43 s vs 313.91 s
- Task 1 token cost
- $0.01541 vs $0.03902
- Task 2 time
- 175.14 s vs 188.06 s
- Task 2 token cost
- $0.01698 vs $0.02157
- Final result
- both arms passed both tasks
- Whole trial, 17 query records
- $0.1829at off-peak rates; $0.3659 at peak
- Output tokens observed
- 56,791the configured output cap was not enforced
Verdict
Extra review cost more on both repairs
Tier list Two workflows on the same model, so it ranks no agent against another.
danmanreal.com/tests/deepseek-direct-vs-extra-review
What was tested
On the tier list's bench
- DeepSeek V4.1 Flash · 4 receipts on file
Related
Independent testing. Sponsorships are always disclosed, and verdicts are never for sale.
Disagree? Bring your receipts