AuditTested AIBy DanPublished Tested Updated
Jarvis 27B on my own PC: 30 tokens a second, and it still recalls at 251K
The report
Verdict
About 30 tok/s, recall to 251K
With everything on my two GPUs, Jarvis wrote about 30 tokens a second, read a 110,451-token prompt at 786 tokens a second and found all 3 hidden codes. It also recalled all 3 at 251K tokens, but at that length part of it runs on the CPU and a reply takes minutes.
On the tier list: These runs measure speed and context on my hardware, not coding quality. Jarvis's A tier is my own verdict from using it, not a score from this report.
| Same rig, ctx 131,072, all on GPU | Jarvis 27B (13.44 GB) | Coder390 Q2 (13.49 GB) |
|---|---|---|
| Writing speed, tok/s (3 trials) | 30.85 / 30.79 / 30.86 | 29.79 / 29.47 / 29.12 |
| Reading 110,451 tokens | 786 tok/s | 640 tok/s |
| Writing at 110K depth | 21.2 tok/s | 18.3 tok/s |
| Recall at 110K | 3/3 | 3/3 |
| 12 reasoning prompts | 12/12, 95 s | 12/12, 104 s |
What Jarvis is
Jarvis is one of the two local seats I use every day: Qwen 3.8 27B Uncensored in a 13.44 GB Q3_K_P file, served by llama.cpp on my own PC. The PC has an RTX 5080 with 16 GB, which also drives my Windows desktop, and an RTX 5060 with 8 GB. Nothing leaves the machine.
Speed and long context
With the whole model on the two GPUs at a 131,072-token context, Jarvis wrote about 30.8 tokens a second over three trials. It read a 110,451-token prompt at 786 tokens a second, kept writing at 21.2 tokens a second at that depth, and found all 3 codes hidden in it.
The model's full 262,144-token context also works, and it found all 3 codes at 251K. The catch is memory: only 45 of 66 layers fit on the GPUs, so speed falls to about 5.8 tokens a second, 1.53 deep in the context, and the recall request took 813 seconds. For everyday work the seat runs at 128K, all on GPU.
Against a 27B that claimed to beat Opus 5.5
On October 10 I tested Coder390, a Qwen 3.8 27B fine-tune from an X post that claimed to beat Claude Opus 5.5 and GPT-6 Astra, against Jarvis on speed and context. At the same file size it was close to a tie but slower on every speed measure (table above). The post's 18.3 GB Q4 file didn't fit my GPUs with the desktop margin; with only 43 of 66 layers on GPU it wrote 5.3 tokens a second and took five times as long on the reasoning prompts. The post's benchmark scores were the author's own, run on a 96 GB card, and I didn't test coding quality.
A speed-up that made it slower
On October 16 I tried DFlash 2, a speculative drafter that another engine measured at about 2.5 times faster on this model, though that engine doesn't fit my 16 + 8 GB GPUs. In llama.cpp b11696 it made Jarvis slower: 12 to 22 tokens a second against a 27.7 baseline on the same workloads. The reason is in llama.cpp's source: Jarvis's model is a hybrid with recurrent layers, and the server rolls back and replays its state after every partly accepted draft, so each round cost about 6.5 single-token steps. The launcher stayed unchanged.
My verdict
Jarvis sits in A on my tier list as the king of local models for an affordable gaming PC. That is my judgement from daily use; these runs back the speed and context side of it, not its coding quality.
Limits of this report
- Speed and context only: no coding-quality test. The tier placement is my verdict, not a measured score.
- One session per test on one PC, with my desktop using 1.4 to 2.4 GB of the 5080 during runs. Numbers will differ on other cards.
- The 12 reasoning prompts are easy, checkable ones; they don't test hard benchmark problems.
- Under heavy background load on October 16, the 128K setup left as little as 1,239 MiB free on the desktop GPU, below my 2,048 MiB safety floor.
Follow the lab
New test reports and tier-list changes get announced in my Discord's #news channel. The full videos are on YouTube.
Receipts
DanManREAL · AI lab
Test receipt
Jarvis 27B: Qwen 3.8 27B Uncensored (HauhauCS Aggressive, Q3_K_P GGUF, 13.44 GB)
- Writing speed, everything on GPU
- 30.8 tok/sthree trials at 131,072 context: 30.85, 30.79, 30.86
- Reading a 110,451-token prompt
- 786 tok/sthen writing at that depth: 21.2 tok/s
- Recall: 3 hidden codes
- 3/3 at 110K · 3/3 at 251Kcodes hidden at 10, 50 and 90% depth of a synthetic logbook
- Full 262,144 context
- 5.8 tok/s · 813 s for the recall45 of 66 layers fit on the GPUs; writing deep in the context fell to 1.53 tok/s
- 12 checkable reasoning prompts
- 12/122,716 thinking tokens, 95 seconds in all
- Largest context the fitter chose on GPU
- 113,920 tokens131,072 also ran with the safety margins intact
- DFlash 2 speculative decoding
- slower: 12 to 22 vs 27.7 tok/sOctober 16, llama.cpp b11696; each draft round cost about 6.5 single-token steps
Verdict
About 30 tok/s, recall to 251K
Tier list These runs measure speed and context on my hardware, not coding quality. Jarvis's A tier is my own verdict from using it, not a score from this report.
danmanreal.com/tests/jarvis-27b-local-speed-and-context
What was tested
Related
Independent testing. Sponsorships are always disclosed, and verdicts are never for sale.
Disagree? Bring your receiptsAll reports
- One inbox for my AI coding agents: triage took 34% less time, resuming work did not improveHead-to-headNov 20
- Mythos 5.2: research and security benchmark source reviewPublic benchmarkNov 17
- Fable 5.2: agentic coding benchmark source reviewPublic benchmarkNov 17
- Claude Opus 5.5: public coding benchmark receiptsPublic benchmarkNov 17
- Astra 6.2: new model and public task-evaluation evidencePublic benchmarkNov 17
- Sol 6.2: new model and public coding benchmark evidencePublic benchmarkNov 17
- Gemini 4 Carbon: benchmark source and usage profilePublic benchmarkNov 17
- Qwen 4: Qwen4-Max launch and benchmark sourcesPublic benchmarkNov 17
- Kimi K3.1 Pro: public coding benchmark receiptsPublic benchmarkNov 17
- GLM-5.3: public coding benchmark receiptsPublic benchmarkNov 17
- Gemini 4 Argon: public coding benchmark receiptsPublic benchmarkNov 17
- Claude Sonnet 5.5: public coding benchmark receiptsPublic benchmarkNov 17
- DeepSeek V4.1 Flash: public coding benchmark receiptsPublic benchmarkNov 17
- Muse Wonder Full 1.0: public coding benchmark receiptsPublic benchmarkNov 17
- Grok 4.7: separate release and public coding resultsPublic benchmarkNov 17
- ai-design-check on two of my FABLE screens: 10 warnings, 4 of them false alarmsField testNov 2
- Vercel Agent Desktop used a sample Trip app in 6 actions without moving the mouseField testNov 1
- Cerberus Guard on sample data: one crawler name, two verdicts, and a block earned by behaviorField testOct 31
- GPT-6.1 Terra vs Opus 5.5: three real projects, and the designs I picked blindHead-to-headOct 29
- Steps → Reduced: opening a terminal agent went from 98 seconds to 11AuditOct 28
- One ChatGPT chat directed Opus and Codex at once through DMR_REMOTEField testOct 28
- Claude Duet ran my channel for a week and caught 269 fake AI crawlersField testOct 27
- Kimi K3.1 Pro built my animation family, after a quota stop and 21 minutes of failed requestsField testOct 24
- ChatGPT + Codex built 33 new DMR_REMOTE toolsField testOct 23
- Gemini 4 Argon reviewed a finished episode: 14 clips, one flag, one inconsistencyField testOct 21
- Qwen 3.8 Flash Next on my RTX 5080: 81 tokens a second with 202K tokens readAuditOct 21
- Does an extra AI review step pay off? A small DeepSeek trial said noHead-to-headOct 19
- My real DeepSeek bill: 12.31 billion tokens for $122.28Cost receiptOct 17
- Gemini 4 Argon kept hanging in Gemini CLI. The cause was a 60-second timeoutIncidentOct 14
- I asked 10 AI coding agents for their biggest weaknessSelf-reportOct 14
- Jarvis 27B on my own PC: 30 tokens a second, and it still recalls at 251KAuditOct 10
- My agents' first game benchmark was wrong. Here's how it was caughtIncidentOct 10
- Pixal3D + Blender turned my 2D art into animated 3D charactersField testSep 22
- DeepSeek Flash answered 200 OK and sent nothing for two hoursIncidentSep 14
- Grok, Codex and DeepSeek played WorldOS for Rank 1. Grok won with 98Head-to-headAug 20
- DeepSeek V4 Flash audited 18 of my real coding sessions. Two said 'done' too earlyAuditAug 4