Strata vs Basalt on 32GB VRAM: peer-Ada + Fixture Tuning to 174 tok/s | Chat LLM
The 4060 Ti Learned to Compute: Basalt Overtakes Strata on Our Rig — Evidence and Protocol
Same rig, fresh builds, verified tiers: Basalt reaches 161-174 real tok/s vs Strata's 121.2 (207.5 vs 192.7 bench, 348 structured fixtures) — because one PR makes the 4060 Ti compute, plus a PCIe ×2 discovery and fixture-based tuning.
November 14, 2026
There are two ways to compare local inference engines: read benchmarks, or write them. This comparison comes from an RTX 5080 + RTX 4060 Ti (2x16GB) + 32GB DDR5 rig, with one declared peculiarity: its author has open pull requests on both sides — three on Strata (#4522, #4534, #4549), one on Basalt (#343). Every setup difference is declared.
Before: one more card, zero more compute
Basalt's "peer" kernels had never been compiled for sm_89 — the Ada architecture of the 4060 Ti. In practice the card was just a VRAM tier read over PCIe: storage, not compute. The MoE work rested entirely on the 5080.
PR #343: compiling the peer kernels for Ada
The fork allows sm_89/sm_120 peers, builds the peer/MMQ kernels and the full prompt path for Ada, with clusters/TMA compiled out below sm_90. Result: the 4060 Ti now natively computes MoE expert windows — Q8_1/Q2_0 in dp4a, standard CUDA. A "storage" card becomes a 14.5 GiB compute tier. This single change explains the flip: 50.5 → 164 tok/s in the first session, before any tuning.
The night of tuning: killing crashes by moving the KV
Long-conversation crashes had a surprising cause: the engine reported "other programs took 4.8 GiB" — but no program was running. It was the engine itself: KV beyond cell 30,000, pinned in RAM, ate the watchdog's margin on every long conversation — and the watchdog does not count its own allocations. The differential proof: switching kv-resident to 42000 (KV in VRAM, GPU reserves at 256) without closing a single program took cap-6000 requests from "Russian roulette" to 4/4. The full diagnosis was reported upstream (issue #353).
Tuning by official fixtures: three levers, each measured
The basalt-probe + basalt-bench protocol (conv/struct/prose fixtures, ABAB) revealed then validated three levers:
The 4060 Ti is on a PCIe 4.0 ×2 slot — 3.2 GB/s vs 28.9 for the 5080. Visible in every dashboard (Gen4x2) yet never noticed. Consequence: freeze its adaptation swaps (--peer-adapt-swaps 0) — the card keeps its 11,194 hottest experts and no longer chokes its own bus; the 5080 keeps adapting at full bandwidth.
spec 4 → 8: on structured content (JSON, code, step-by-step reasoning), drafter acceptance exceeds 95% — the longer verification window averages 6.8 tokens per cycle: structured fixtures jump from 237 to 348 tok/s (+47%).
Pool fix PR #328 (a wakeup lost in the CPU worker barrier): +11% on prose — 10ms micro-stalls disappear.
And a useful counter-result: KV NVFP4 (#314, 0.55 GiB) loses against int8 at 65K context (297 vs 348 structured) — it remains the weapon for 131K+ contexts.
The numbers — fresh builds, same Nov 14 morning, long-thinking workload
Real throughput (technical prompt, reasoning, temp 0): Strata 0.1.59 3-PR 121.2 tok/s — Basalt final 161-174 tok/s.
Official fixtures (conv/struct/prose): 157 / 348 / 146 tok/s.
Prefill 16K: 1,393 tok/s.
Final recipe: kv int8 elastic (all in VRAM), spec 8, pool 7, peer-adapt-swaps 0 (4060 Ti is PCIe ×2), budget 8.15, reserves 256/128. Sequence cap-6000+image+text passes 5/5 (174/161/164/161 t/s).
The duel protocol (what makes the numbers comparable)
Same evening, same machine (active 512MB iGPU, 30.9 GB RAM), both engines compiled the same day.
Two labeled content types: counting (drafter acceptance ~99%) and real-reasoning (59-66%) — both reported, never mixed.
Cold and warm distinguished; launch order is a variable (both engines freeze their tier on free RAM at boot).
Tier line verified at boot ("file (SSD) 0.0 GiB" = valid); 2GB RAM guard = anti-death floor.
Memory: -33%, and fully resident
Where the reference Strata config deploys a larger 2-bit quant (~44 GiB, partial offload), Basalt final fits in 31 GiB, entirely resident. Less model RAM, less PCIe, more local compute.
The code bench: the 1-bit that beats the 2-bit
On our 34-exercise code suite (polyglot, temp 0, pytest as judge), same base model: Basalt IQ1_S 15/34 — 75.7% of tests passed, the family's best score on this suite — vs 8/34 for Q2_0 on Strata. Declared: the auto thinking budget is larger on Basalt (~34,500 vs 4096); the equal-budget duel is yet to be run.
Endurance: the non-comparable axis (assumed)
Strata has been running in production for weeks. Basalt final has a dedicated stress test: 81 minutes, 67/68 requests at a 46K cap. These two facts do not compare — and the article says so.
Verdict by use case
Real long-thinking throughput on this rig: Basalt final, 161-174 vs 121.2 tok/s (Strata 3-PR, same morning, equal tier).
Proven long-run production: Strata, today.
The lessons go beyond both engines: check that your kernels compile for your second card's architecture — and read the PCIe line in your dashboards: a discrete ×2 slot limited this card for weeks without anyone looking.
Reproducing
Basalt final (peer-ada fork, PR #343): basalt_serve.json — model Qwen3.8-FN-UD-IQ1_S on GPU 0,1, kv int8, max-context 65536, vision dynamic; engine: resident-experts true, pool-workers 7, spec 8, expert-cache auto, prefill auto, resident-budget-gib 8.15, peer-adapt-swaps 0, vram-reserve-mib 256, peer-reserve-mib 128. Verify at boot: "experts live in ... file (SSD) 0.0 GiB" and "RAM X GiB free, guard 2.00 GiB".
