Maximising Qwen3.8-27B on a 5090 Laptop

If you read one thing, read this. Every flag is justified by a measurement below.

llama-server -m Qwen3.8-27B-UD-Q4_K_M.gguf \
  -ngl 999 -fa on --jinja \
  -c 262144 -b 1024 -ub 1024 \
  --cache-type-k q4_0 --cache-type-v q4_0 \
  --spec-type draft-mtp --spec-draft-n-max 5 \
  --reasoning-effort low \
  --parallel 1 --no-mmap \
  --host 127.0.0.1 --port 8080

Each of these was a fork where the obvious choice was wrong.

Every failure encountered, with its exact symptom. Ordered by cost.

Trap 13 in detail — how a fake flag was caught

--reasoning-effort high returns plausible numbers, so it looks like data. Stripping timing lines and hashing the output bodies proves all three are the same generation:

high.log     14190 bytes   64.0 t/s   md5 4df134687af5408a19b989dc4b5c8530
xhigh.log    14190 bytes   63.7 t/s   md5 4df134687af5408a19b989dc4b5c8530
default.log  14190 bytes   63.9 t/s   md5 4df134687af5408a19b989dc4b5c8530

The 0.3 tok/s spread is run noise on identical work. Generalisable lesson: when a flag has no documented value list, verify it changed the output before you trust the number.

Individual runs, not just means. --temp 0 and a fixed seed make speculative decoding deterministic run-to-run, but NOT byte-identical across draft widths — see the corrected methodology note below.

Peak VRAM at 200K filled (23,514 MB) matches the allocated-only figure (23,504 MB) almost exactly, confirming the KV cache is preallocated rather than grown. VRAM planning can use the empty-window numbers; throughput planning cannot. Note also that Experiment D's “Prompt t/s” column (~100–111) is overhead on a tiny prompt, not a real prefill rate — the true rates are in Experiment F.

Three things worth extracting. The optimum moves with output shape — n5 for code and prose, n10 for JSON and HTML. The gain varies by 5×, from +45% on prose to +250% on JSON; rigid formats are trivially predictable, so drafts land far more often. And n4 on prose is actively harmful (38.6 against a 40.4 baseline) — the only measured config anywhere in this guide that is slower than no speculation at all.

141.6 tok/s on JSON is the fastest figure measured on this box, well above the 96.8 headline, which was code-shaped. If your workload is structured extraction or API-shaped output, the ceiling here is considerably higher than the front page suggests.

Sampled at 2 Hz during Experiment B, with the limit raised to 175 W.

Three things fall out of this. Average draw is 95–109 W against a 170 W ceiling — the GPU is not power-starved even at the 95 W default, which is why raising the limit changed nothing. Peak temperature is 58 °C, so thermal throttling is simply not a factor on this chassis; the common advice to power-cap for sustained throughput does not apply here. The fastest config draws the least power — n-max 5 averages 95.0 W, the lowest of the set, while being the fastest. Speculative decoding replaces memory traffic with cheap parallel verification.

Two methodology errors were made and corrected during this work. Both are easy to repeat and neither announces itself.

Benchmark harness

$common = @('-m',$model,'-ngl','999','-fa','on',
            '-ctk','q4_0','-ctv','q4_0',
            '-c','32768','-n','1024','--seed','42','--temp','0',
            '-st','--no-warmup','--reasoning','off','-p',$prompt)

foreach ($n in 2,3,4,5,6,7,8,10,12) {
  for ($i=1; $i -le 2; $i++) {
    & $cli @common --spec-type draft-mtp --spec-draft-n-max $n *> "$out\n$n-$i.log"
  }
}
# llama-cli reports: [ Prompt: NNN t/s | Generation: NN.N t/s ]
# NOT the llama_perf_context_print / "eval time" format older guides parse.

Stated so nobody mistakes absence of data for absence of a problem.

• Filled-context throughput is unmeasured. Now measured — Experiment F. Decode retains 43% at 200K filled (36.1 tok/s). Better than the ~12 tok/s some published far-context figures suggest, but still below our unoptimised baseline.
• Depth beyond 200K is still unmeasured. The 262,144 window was allocated but filled only to ~200K. The last 60K may degrade further.
• Single-shot only. Each measurement is one prompt, one generation. A real multi-turn session with a growing cache may behave differently again.
• Quality was never measured. No perplexity, no benchmark scoring. The claim that -effort low is a reasonable default rests on it completing sensibly, not on evidence it answers as well as xhigh.
• Only one quant tested. Q5_K_M and Q6_K were never pulled. The speed cost of higher fidelity on this box is unknown.
• One prompt shape. Now measured — Experiment G. The optimum does move: n5 for code and prose, n10 for JSON and markup. Translation and tool-calling remain untested.
• Acceptance rates are derived, not measured. Inferred from speedup ratios.
• The n4 memory downclock was seen once. Unexplained, possibly relevant to n4's underperformance.
• Sysmem fallback was never actually disabled during these runs — it is a Control Panel setting. Results held anyway because 256K fits, but a filled window may behave differently.

The laptop 5090 is a different part from the desktop 5090 — 24 GB on a 256-bit bus at 896 GB/s, against 32 GB / 512-bit / 1792 GB/s. Roughly half the bandwidth and three-quarters the VRAM. Published desktop figures do not transfer, and neither do these in the other direction.

Build flags of record

-DCMAKE_BUILD_TYPE=Release
-DGGML_CUDA=ON
-DCMAKE_CUDA_ARCHITECTURES=120
-DGGML_CUDA_FORCE_CUBLAS=OFF
-DGGML_CUDA_F16=ON
-DLLAMA_CURL=OFF