Xyntetik xyntetik

Evidence

Every claim, with the measurement behind it.

This page collects what has actually been measured, when, on what, and where the raw material lives. Results that went against the project are on it too. If a claim you find elsewhere is not backed here or in the repository, treat it as unproven.

One tool call cut short at token budgets 1 to 16: which engine still hands back a call that runsmax_tokens123581664RunnervLLMllama.cpp{ "Ollama500TensorRT-LLMSGLang

One tool call cut short by the token limit, six engines, budgets 1 to 64. Only Runner hands back a call that runs below the control. Rung by rung, 2026-08-19.

The four results that matter most, losing rows first. Each number links to its section and its raw material.

6 to 10%
of llama.cpp's prompt-processing speed on CUDA, the project's weakest row; decode reads 69 to 93% on the same files
Same box, same GGUF files, full offload verified. 2026-09-02 and 2026-10-03.
1 of 6
engines that hand back a runnable tool call when the token limit cuts it short, at every budget from 1 to 16 tokens
Runner, vLLM, llama.cpp, Ollama, TensorRT-LLM, SGLang. 2026-08-19.
21 of 22
rows where Runner's output is closer to the model publisher's own float32 implementation than llama.cpp's, across twelve families
2,000 teacher-forced positions per row. 2026-09-07.
0.69 to 1.00
held-out tool-call score after training through the 4-bit file it serves; two runs wrote the same adapter bytes
Qwen3-4B Q4_K_M, reproduced by an outside tester on a Tesla T4. 2026-08-22.

Truncation recovery

Six engines, seven token budgets, one that closes the document.

Claim under test: tool calls survive the token limit. Everything that could move a verdict is pinned as data in the probe: the tool schema (get_weather(city, units), both required, no additional properties), the prompt, tool_choice: required, temperature 0, budgets 1, 2, 3, 5, 8, 16 and 64. The 64-token control must complete on every engine, which proves the small rungs measure truncation and not misconfiguration.

executable tool call, arguments parse no usable call
Per token budget, does the client receive an executable tool call123581664 ctrlRunnerRunner, max_tokens 1: executable tool call, arguments parseRunner, max_tokens 2: executable tool call, arguments parseRunner, max_tokens 3: executable tool call, arguments parseRunner, max_tokens 5: executable tool call, arguments parseRunner, max_tokens 8: executable tool call, arguments parseRunner, max_tokens 16: executable tool call, arguments parseRunner, max_tokens 64: executable tool call, arguments parsevLLM 0.27.1vLLM 0.27.1, max_tokens 1: no usable callvLLM 0.27.1, max_tokens 2: no usable callvLLM 0.27.1, max_tokens 3: no usable callvLLM 0.27.1, max_tokens 5: no usable callvLLM 0.27.1, max_tokens 8: no usable callvLLM 0.27.1, max_tokens 16: no usable callvLLM 0.27.1, max_tokens 64: executable tool call, arguments parsellama.cpp b10488llama.cpp b10488, max_tokens 1: no usable callllama.cpp b10488, max_tokens 2: no usable callllama.cpp b10488, max_tokens 3: no usable callllama.cpp b10488, max_tokens 5: no usable callllama.cpp b10488, max_tokens 8: no usable callllama.cpp b10488, max_tokens 16: tool_calls present, arguments do not parse{ "llama.cpp b10488, max_tokens 64: executable tool call, arguments parseOllama 0.32.14Ollama 0.32.14, max_tokens 1: no usable callOllama 0.32.14, max_tokens 2: no usable callOllama 0.32.14, max_tokens 3: no usable callOllama 0.32.14, max_tokens 5: no usable callOllama 0.32.14, max_tokens 8: no usable callOllama 0.32.14, max_tokens 16: HTTP 500500Ollama 0.32.14, max_tokens 64: executable tool call, arguments parseTensorRT-LLM 1.2.1 †TensorRT-LLM 1.2.1 †, max_tokens 1: no usable callTensorRT-LLM 1.2.1 †, max_tokens 2: no usable callTensorRT-LLM 1.2.1 †, max_tokens 3: no usable callTensorRT-LLM 1.2.1 †, max_tokens 5: no usable callTensorRT-LLM 1.2.1 †, max_tokens 8: no usable callTensorRT-LLM 1.2.1 †, max_tokens 16: no usable callTensorRT-LLM 1.2.1 †, max_tokens 64: executable tool call, arguments parseSGLang 0.5.17 †SGLang 0.5.17 †, max_tokens 1: no usable callSGLang 0.5.17 †, max_tokens 2: no usable callSGLang 0.5.17 †, max_tokens 3: no usable callSGLang 0.5.17 †, max_tokens 5: no usable callSGLang 0.5.17 †, max_tokens 8: no usable callSGLang 0.5.17 †, max_tokens 16: no usable callSGLang 0.5.17 †, max_tokens 64: executable tool call, arguments parse
Measured 2026-08-19 on granite-4.1-3b Q4_K_M (Runner and llama.cpp on CPU, Ollama on GPU from the same GGUF, vLLM from the HF safetensors on GPU). † TensorRT-LLM and SGLang on a Qwen3-1.7B substitute because their registries did not carry granite; SGLang on an RTX 3070 under WSL2. Runner c7d6c3d (0.1.19-alpha), vLLM 0.27.1 with xgrammar 0.2.3, llama.cpp b10488, Ollama 0.32.14, TensorRT-LLM 1.2.1, SGLang 0.5.17 with xgrammar 0.2.1.
What each engine hands the caller, rung by rung
max_tokensRunnervLLMllama.cppOllamaTensorRT-LLM †SGLang †
1parsesnone; leaks <tool_call> into contentnone; leaks <tool_call> into contentnone; empty contentnone; leaks bare tagnone; leaks bare tag
2parsesnone; leaknone; leaknone; emptynone; emptynone; empty
3parsesnone; leaks {"none; leaks {"none; emptynone; emptynone; empty
5parsesnone; leaks {"name":none; leaks {"name":none; emptynone; emptynone; empty
8parsesnone; leaks {"name": "get_weathernone; leaks {"name": "get_weathernone; emptynone; emptynone; empty
16parsesnone; leaks {"city": "tool_calls present, arguments {"city": " do not parseHTTP 500none; emptynone; empty
64parses, completesparses, completesparses, completesparses, completesparses, completesparses, completes

The property is an engine guarantee, grammar plus closer, not model quality: it holds identically for the random two-layer CI fixture and for granite-4.1-3b, which is why make test-truncation can run on any CPU with no GPU and no competitor, and does on every release. The claim covers the engines and versions measured, not engines that were not. Method, the substitute-model notes and the raw responses: docs/truncation-benchmark.md, tests/torture/truncation/2026-08-19-granite-4.1-3b/. The agent-torture gate tests the same failure inside multi-turn loops.

Throughput

Speed against llama.cpp, same files, same box.

One machine, one method, both engines on the same GGUF files. Runner via --bench-json -n 128 -b 64 (mean of 2), llama.cpp via llama-bench -p 512 -n 128 -ngl 99 -r 2, full GPU offload verified on every row. Absolute numbers depend on the hardware; the ratios are the result, and even those shift with the compute-to-bandwidth balance.

Runner throughput as a percentage of llama.cpp, decode and prefillDecode tokens/s, runner as % of llama.cpp0%25%50%75%100%Llama-3.2-3B Q4_K_MLlama-3.2-3B Q4_K_M: runner 87.9 vs llama.cpp 101.8 tok/s (86%)86%Phi-4-mini Q8_0Phi-4-mini Q8_0: runner 53.8 vs llama.cpp 57.7 tok/s (93%)93%granite-3.3-8b Q4_K_Mgranite-3.3-8b Q4_K_M: runner 39.7 vs llama.cpp 45.4 tok/s (87%)87%granite-4.1-8b Q4_0granite-4.1-8b Q4_0: runner 41.7 vs llama.cpp 46.5 tok/s (90%)90%Phi-3.5-mini Q4_K_MPhi-3.5-mini Q4_K_M: runner 72.0 vs llama.cpp 83.2 tok/s (87%)87%gemma-4-12B Q4_K_Mgemma-4-12B Q4_K_M: runner 23.2 vs llama.cpp 28.8 tok/s (81%)81%Qwen3-30B-A3B Q4_K_M (MoE)Qwen3-30B-A3B Q4_K_M (MoE): runner 62.2 vs llama.cpp 86.6 tok/s (72%)72%gemma-4-26B-A4B Q4_0 (MoE)gemma-4-26B-A4B Q4_0 (MoE): runner 23.3 vs llama.cpp 56.2 tok/s (41%)41%Qwen2.5-32B Q3_K_SQwen2.5-32B Q3_K_S: runner 1.8 vs llama.cpp 14.3 tok/s (13%)13%Prefill tokens/s, runner as % of llama.cpp0%25%50%75%100%Llama-3.2-3B Q4_K_MLlama-3.2-3B Q4_K_M: runner 525.7 vs llama.cpp 6387.6 tok/s (8%)8%Phi-4-mini Q8_0Phi-4-mini Q8_0: runner 355.6 vs llama.cpp 5960.6 tok/s (6%)6%granite-3.3-8b Q4_K_Mgranite-3.3-8b Q4_K_M: runner 220.5 vs llama.cpp 2288.9 tok/s (10%)10%granite-4.1-8b Q4_0granite-4.1-8b Q4_0: runner 151.1 vs llama.cpp 2359.2 tok/s (6%)6%Phi-3.5-mini Q4_K_MPhi-3.5-mini Q4_K_M: runner 284.9 vs llama.cpp 5120.3 tok/s (6%)6%gemma-4-12B Q4_K_Mgemma-4-12B Q4_K_M: runner 123.7 vs llama.cpp 1439.2 tok/s (9%)9%Qwen3-30B-A3B Q4_K_M (MoE)Qwen3-30B-A3B Q4_K_M (MoE): runner 128.5 vs llama.cpp 2326.3 tok/s (6%)6%gemma-4-26B-A4B Q4_0 (MoE)gemma-4-26B-A4B Q4_0 (MoE): runner 34.3 vs llama.cpp 2429.7 tok/s (1%)1%Qwen2.5-32B Q3_K_SQwen2.5-32B Q3_K_S: runner 15.3 vs llama.cpp 469.0 tok/s (3%)3%
MIG 1g.24gb slice of an RTX PRO 6000, runner 6d705e9 vs llama.cpp b10353 CUDA, measured 2026-09-02. The box carried a load average of 18 from other work during this run, which lowered both sides' absolute numbers against August; the ratios are the result. Source: docs/benchmarks.md.
Tokens per second: the 1.0.0 re-measure and all three hosts of 2026-09-02
host and datemodeldecode runner / llama.cppprefill runner / llama.cpp
RTX 3070 8 GB, Windows 11, runner 1.0.0 vs llama.cpp b10754, 2026-10-03Qwen2.5-7B Q4_K_M64.8 / 81.7 (79%)322.7 / 3445.8 (9.4%)
granite-4.1-3b Q8_074.9 / 92.6 (81%)515.4 / 5539.5 (9.3%)
Qwen2.5-3B Q4_K_M112.8 / 159.5 (71%)622.6 / 6229.3 (10.0%)
Llama-3.2-3B Q4_K_M109.2 / 158.4 (69%)596.3 / 6691.2 (8.9%)
granite-4.2-8b Q4_K_M52.0 / 68.4 (76%)265.5 / 2758.1 (9.6%)
granite-4.2-3b IQ3_S19.0 / 154.1 (12%)469.6 / 5315.6 (8.8%)
MIG 1g.24gb, llama.cpp b10353, 2026-09-02Llama-3.2-3B Q4_K_M87.9 / 101.8 (86%)525.7 / 6387.6 (8.2%)
Phi-4-mini Q8_053.8 / 57.7 (93%)355.6 / 5960.6 (6.0%)
granite-3.3-8b Q4_K_M39.7 / 45.4 (87%)220.5 / 2288.9 (9.6%)
granite-4.1-8b Q4_041.7 / 46.5 (90%)151.1 / 2359.2 (6.4%)
Phi-3.5-mini Q4_K_M72.0 / 83.2 (87%)284.9 / 5120.3 (5.6%)
gemma-4-12B Q4_K_M23.2 / 28.8 (81%)123.7 / 1439.2 (8.6%)
Qwen3-30B-A3B Q4_K_M (MoE)62.2 / 86.6 (72%)128.5 / 2326.3 (5.5%)
gemma-4-26B-A4B Q4_0 (MoE)23.3 / 56.2 (41%)34.3 / 2429.7 (1.4%)
Qwen2.5-32B Q3_K_S1.8 / 14.3 (13%)15.3 / 469.0 (3.3%)
RTX 3070 8 GB, Windows 11, llama.cpp b10754 CUDA 13.3, 2026-09-02Qwen2.5-7B Q4_K_M65.6 / 81.6 (80%)112.3 / 2224.5 (5.0%)
granite-4.1-3b Q8_078.6 / 94.3 (83%)493.5 / 5614.1 (8.8%)
Qwen2.5-3B Q4_K_M113.3 / 161.2 (70%)234.0 / 6069.9 (3.9%)
Apple M1 8 GB, Metal, llama.cpp Homebrew ggml 0.22.0, 2026-09-02granite-4.1-3b Q8_011.6 / 14.4 (80%)47.8 / 266.5 (17.9%)
gemma-3-4b Q4_K_M8.4 / 20.3 (41%)62.9 / 250.3 (25.1%)

Decode: parity is the physical target

Single-stream decode is memory-bandwidth-bound on the slice; both engines read the same quantized weights per token. Dense decode moved from 73 to 79% in July to 81 to 93% in September through decode GEMV work and a KV-ring accounting fix. On the RTX 3070 the 1.0.0 binary reads 69 to 81%; the one i-quant row, granite-4.2-3b at IQ3_S, decodes at 12% where the same family at Q4_K_M reads 76%, and that gap is open work.

Prefill: llama.cpp's win, published as such

llama.cpp's prefill comes from a mature tensor-core GEMM stack across every quant. Runner's tensor-core path is held to a teacher-forced tolerance gate (at most a near-tie flip or two in 64 positions, at most 0.012% mean logit deviation on the k-quant rows and 0.09% on the 1-bit codebook rows promoted in September 2026, free-running text identical) and its tile is 64 columns wide. The gap is reported, not hidden.

The M1 rows show the coexistence trade from the losing side

Measured with 1.3 GB of RAM available, Runner logged its own eviction warning and paged while llama.cpp wired the model in. The gemma row is that effect, not a kernel result, and the document says to re-measure on a quiet machine before reading it as one.

A July table in the same document was taken before a CUDA prefill correctness bug (uninitialized shared memory for token columns 16 to 63) was found; it is kept as history and superseded by the August re-measurement on the fixed kernel. The benchmark's own control run also found one silent MoE GPU-to-CPU fallback, since fixed and guarded by a test.

Quantization fidelity

Constrained decoding guarantees the shape of a tool call at any quantization. Not its contents.

A full quant ladder, one Q8_0 reference against several variants, 16 agent-torture requests at temperature 0, with a zero-point self-check that two independent reference spawns agree on every call at 0 KLD before anything is measured. Three axes: does the output validate against the schema, did the variant call the same function as the reference, and do the parsed arguments match exactly.

granite-4.1-3b, argument agreement Hermes-4-14B, argument agreement schema conformance and tool selection, both families
Tool-call fidelity down the quantization ladder: shape holds at 100 percent, argument agreement decays40%60%80%100%Q8_0Q6_KQ5_K_MQ4_K_MQ4_0schema conformance and tool selection: 100% on every rung, both familiesgranite-4.1-3b Q8_0: argument agreement 100.0%granite-4.1-3b Q6_K: argument agreement 64.3%granite-4.1-3b Q5_K_M: argument agreement 57.1%granite-4.1-3b Q4_K_M: argument agreement 57.1%granite-4.1-3b Q4_0: argument agreement 50.0%granite-4.1-3b50% at Q4_0Hermes-4-14B Q8_0: argument agreement 100.0%Hermes-4-14B Q6_K: argument agreement 78.6%Hermes-4-14B Q5_K_M: argument agreement 78.6%Hermes-4-14B Q4_K_M: argument agreement 64.3%Hermes-4-14B Q4_0: argument agreement 50.0%Hermes-4-14B50% at Q4_0
granite-4.1-3b measured 2026-08-14; Hermes-4-14B (bartowski's ladder, hashes verified against the source repository) measured 2026-08-15. Schema conformance and tool selection held at 100% on every rung of both ladders; argument agreement decayed to 50% at Q4_0 on both. Source: docs/quant-fidelity.md.

Whole-model fidelity has a size threshold

Mean KLD against the model's own reference at Q4_K_M, under the adopted bar (margin-qualified top-1 at least 97% and mean KLD at most 0.05 against the named parent, 400 teacher-forced positions). Below about 8B parameters, uniform 4-bit fails the bar; from 8B up it passes. That threshold, and not a blanket "4-bit is unshippable", is what the ladder found.

model at Q4_K_Mparamsmargin-qualified top-1mean KLDverdict
gemma-4-E2B~5B MoE, 2B active85.00%0.3624fail
granite-4.1-3b3B93.00%0.1376fail
Phi-4-mini3.8B96.50%0.1258fail
Qwen3-8B8B100.00%0.0419pass
Hermes-4-14B14B99.75%0.0279pass

Training

One base, one task, every number on a public artifact.

Held-out exact tool call score before and after training, and after merging0.000.250.500.751.00Base Qwen3-4B Q4_K_MBase Qwen3-4B Q4_K_M: exact-call score 0.690.69Base + adapter (--lora)Base + adapter (--lora): exact-call score 1.001.00Merged into F16Merged into F16: exact-call score 1.001.00Merged into Q8_0Merged into Q8_0: exact-call score 1.001.00Merged back into Q4_KMerged back into Q4_K: exact-call score 0.690.69 erased
Held-out exact tool-call score, Qwen3-4B. Measured 2026-08-22. Artifact: Qwen3-4B-Runner-ToolUse-Q4_K_M; design, gates and failure modes in docs/adaptation-engine.md.
base

Qwen3-4B Q4_K_M, the frozen 4-bit serving weights

training path

directly through the quantized inference artifact, on the CPU

held-out exact call

0.69 before, 1.00 after

reproducibility

two independent runs, same sha256 on the adapter file

precision study

adapters trained through BF16 and Q8_0: cosine 0.9998; through Q4_K_M: 0.9926. Measurably different objects, capability-equivalent on this task's supervised decisions. An external 36-prompt boundary bank finds 3 of 36 prompts where the three adapters choose different tools; a property of that bank, not a rate

neutral-corpus drift

nll per token 4.063 to 4.026, unrelated text left alone

merge study

into Q8_0 or F16 keeps the 1.00 (verified in stock llama.cpp); into the 4-bit base erases it, 0.69 again, 98.55% of weight bytes round back to the base's codes. Survival is monotone in delta magnitude: erased through 2x, partial at 4x, full at 8x

interop

the adapter scores the same 1.00 served by stock llama.cpp; a community F16 adapter loads back into Runner, which also found and fixed the F32-only loader gap

scope

one base, one task. The 8B and 14B ladder is open work and these numbers are not a promise about other families

Terminal recording: two independent training runs print the same adapter sha256
Two independent training runs producing byte-identical adapters. Recording from the repository README.
Terminal recording: merging the adapter into the 4-bit base returns the base score; the 8-bit merge keeps it
Merging the adapter into the 4-bit base erases it; 8-bit keeps it. Recording from the repository README.

Receipts

A recorded run is a signed record, and a verifier answers with one exit code.

--transcript records model, adapter and binary hashes, the effective execution profile, seed and sampling configuration, prompt and output token ids and the exact streamed output bytes. --verify replays it: VERIFIED (0), DIVERGED at a token or output byte (2), UNVERIFIABLE for an invalid record or an artifact mismatch (3). Receipts chain, are signed with an Ed25519 key you generate, and the loaded model can be verified against an OpenSSF Model Signing bundle.

Receipts and the determinism scope
Claimed, gated in CI

Same executable, same inputs, same sampled tokens, across runs and thread counts. Training byte-determinism on the same build. Quantize and merge write the same bytes for the same inputs on the same build.

Replay tiers

T1, same binary: bit-exact replay. T2, cross-ISA: token-level replay on another machine; floating-point internals may differ and the boundary is libm.

Explicitly not claimed

Independent rebuilds are not byte-identical. Cross-engine token identity is not promised. Numerical identity of hidden states is never promised. Timing and throughput are not part of the contract.

First external reproduction

Adapter byte-equality was reproduced independently on a Tesla T4 by a Hugging Face community tester and is credited in the docs.

Compatibility gates

Pinned files, declared checks, executed on every release.

The machine-readable manifest tests/compatibility/models.json pins files by SHA-256 and declares each check independently. The release check refuses a report with zero executed rows, and the CUDA smoke gate on a second device family is mandatory before a tag.

Hosted CI has no GPU, so it cannot say when a claimed device last ran the gates. docs/device-evidence.json records that per device class, written from the runner's own --caps on the machine in question rather than typed by hand, and the release check refuses a tag while one of them is stale. The gap it exists to close was real: the Windows suite went nineteen days and eight failures before anybody looked. The half CI can reach now runs there too, nightly and on every push to the main branch, rather than as a hand-picked selection of smoke tests.

load

The pinned file hashes and loads.

tokenizer

The committed 721-string corpus is compared with the model's Hugging Face tokenizer; revision-bound captures make declared rows replayable offline.

greedy_reference

Greedy tokens are compared with a pinned llama.cpp revision.

cpu_cuda

CPU and CUDA scalar-path greedy output are compared; byte-identity is the contract on the scalar path.

chat

A real Chat Completions request answers through the model template.

Cross-engine identity, stated with its envelope

Agreement with another engine, not fidelity: where a publisher reference was measured it comes first. Nemotron-Nano-9B-v2 Q8_0: 100% margin-qualified top-1 against NVIDIA's float32 reference over 100 positions, then 5 of 6 greedy completions byte-identical to llama.cpp b10353, the miss a single-token near-tie. granite-4.0-h: 94.36% margin-qualified at Q4_K_M against IBM's reference on the micro member (llama.cpp 93.67%), then token-identical to llama.cpp on deterministic prompts on h-small. Nemotron-3.5-Lightning-30B: no publisher reference yet, 4 of 5 against llama.cpp. Each envelope is published with the artifact, not rounded up.

Method: docs/compatibility-program.md; reports: docs/compat-reports/; measured-envelope convention: README.

Artifact ledger

Published only after the stated gate. Kept published when the gate is missed.

Every fidelity claim below is measured under the adopted dual-column bar: margin-qualified top-1 at least 97% and mean KLD at most 0.05 against the named parent over 400 teacher-forced positions, zero point exact, with plain top-1 always reported beside it. Read each card's provenance before treating a derivative as equivalent to an original checkpoint.

  • Xyntetik-Kvist-14B, GGUF, the training record and the research checkpoints
    Built for one job, tool-calling agent loops on the runner from a 24 GB card, and not a drop-in replacement for Muse-Glimmer-30B or any general-purpose model (held-out KLD 0.762 against its own parent). Not a derivative and not under the fidelity bar: a dense 14B student in the Muse-Glimmer architecture, distilled from Muse-Glimmer-30B, released 2026-09-28 under a preregistered five-arm gate. On the gate's 60 held-out closed-loop tool tasks it solves 57 where the parent solves 60 and the untrained control 0. The card leads with its disclosures: calc is the weak kind (12 of 15 over 160 tasks), 7 of 160 runs end in a reasoning loop, twelve gated attempts for two full passes. Pins Runner v0.5.7. The research checkpoints (the end of general distillation and attempt 11, weights only) are there to resume the training from, not to serve.
  • The typed-decisions record
    Research evidence, not a model and no weights: TD-000 and TD-001 on whether Kvist-14B's /v1/decide probabilities can be used as decisions. TD-000 measured three models' probabilities and tested nothing. TD-001, preregistered, asked whether a log-loss LoRA of Kvist-14B beats a per-class temperature map by 0.05 nats on held-out ledger questions, and came out inconclusive (-0.074 nats, 95% interval [-0.128, -0.037]); the adapter is not distributed. The questions, every scored row, the preregistrations and the pinned analysis are there. Kvist collection.
    research record
  • Qwen3-30B-A3B selective precision
    Attention Q8_0, experts Q4_0, 17.99 GB, from a byte-verified first-party Q8_0 source. Passes the bar where the official uniform Q4_K_M fails it. Built with --type-plan; the plan is on the card. The artifact class the project leads with.
    passes bar
  • Qwen3-Coder-30B keep-120
    Expert-pruned, 17.5 GB. Passes both the original and the current bar; the only published artifact to clear the original bar unaided.
    passes bar
  • Qwen3-4B Runner ToolUse adapter
    The LoRA behind the training claim, produced twice with the same sha256. Card carries the sidecar and the training command.
    reproduced
  • gemma-4-31B-it, attention block 48 removed
    17.58 GB Q4_0. One attention sublayer physically dropped from Google's QAT Q4_0 release with --remove-sublayer; frees 64 MiB of KV at 4k and 512 MiB at 32k plus 74.3 MB of file; tracks its parent at KLD 0.0223 on 44,413 held-out positions. Loads only in Runner 0.4.7 and later, CPU path; llama.cpp refuses it by name, and the card says so.
    KLD 0.0223
  • Muse-Glimmer-30B surgical, Q4_K and Q8_0
    The decoder cut and healed (4.75% removed in the Q4_K file, 14.61 GB; 6.34% in the Q8_0, 27.58 GB), quantized by Runner's own quantizer. Both pass the bar against the BF16 parent (97.74% / 0.04810 and 98.34% / 0.03883). The Q4_K cost was predicted to 0.6% before the file was built. Provenance and checksums in the repository; the surgery report carries the method.
    passes bar
  • Qwen3.8-27B GSQ-RCO IQ3_S, recovered scales
    11.77 GB. ISTA-DASLab's IQ3_S file with every quantised block's fp16 scales retrained by distillation against the BF16 parent: same integer codes, layout and byte length. Mean KLD 0.0450, margin-qualified top-1 97.80% on 500 held-out positions, inside the bar. The unmodified source reads 0.0480 and 97.60% on the same positions, so the edge over it is a consistent direction that this sample does not resolve. Run record, logs, fidelity and HellaSwag records and code in the scale-recovery dataset.
  • Muse-Glimmer-30B surgical 6.34%, Q4_K with recovered scales
    14.61 GiB. The healed 6.34% model failed at plain Q4_K (0.05815) and passes with retrained scales (0.04949, 97.90% on the 45,056-position held-out split). Preregistered before measurement; the four-arm scores, logs and code are in the scale-recovery dataset.
  • NVIDIA-Nemotron-Nano-9B-v2 Q8_0
    Not a bar-gated derivative: the first Mamba-2 hybrid artifact Runner supports, a near-lossless Q8_0 of NVIDIA's base: 100% margin-qualified top-1 against NVIDIA's float32 reference over 100 positions (llama.cpp also 100%), and 5 of 6 greedy token-identical vs llama.cpp b10353. NVIDIA Open Model License.
    support artifact
  • gemma-4-E2B-it Q4_K_M/Q4_0 mix
    2.63 GB smoke-test artifact. Fails the fidelity bar, the card carries the dual-column numbers, and it remains the fastest way to try Runner on an 8 GB machine.
    fails bar
  • gpt-oss-20b keep-30 MXFP4
    11.5 GB, 32-to-30-expert derivative. Its originally published number did not reproduce; the card leads with the measured status. Kept as a near-miss with its numbers in the open.
    fails current bar
  • Qwen3.8-Flash-Next MTP head
    4.17 GB Q8_0, the draft head the model's release trunk shipped without, built from the publisher's BF16 tensors with every source tensor's sha256 in the header. Not a model: it drafts beside the ISTA-DASLab IQ3_S trunk and the trunk verifies, so only speed changes. llama.cpp b11433 loads it as -md … --spec-type draft-mtp (CPU-only decode 12.5-13.2 to 17.0-20.4 tok/s, 69-82% of drafts accepted); Runner 1.1.0 with --mtp-file, byte-identical text, no CPU speed gain yet (the card says why).
  • Evidence datasets over third-party files, no weights republished
    Hermes-4-14B quant fidelity, the Qwen3 speculative pair, the granite-4.0-h-small and Nemotron-3.5-Lightning support reports, and two frontier reports where nothing cleared the bar and beat upstream: the Lightning prune frontier and the Muse-Glimmer quant frontier, where Meta's own Q4_K_M passes and six Runner plans did not beat it, stated openly.
    reports
  • Xyntetik-Blad-Genesis-0.7M and the training record
    A research artifact at toy scale, not a Runner model: no GGUF, PyTorch on CPU only, with a small run_example.py. 686,756-parameter program-proposal cores (3 seeds x 160, 320 and 640 iterations) trained from scratch with no human text. A tiny model trained only on its own executed programs beat the best blind search at equal executions on a synthetic list DSL; every self-improvement mechanism tested failed once its cost was counted (one host). The record is the full preregistered GENESIS I record (18 studies, 1,407 files, 655 MB); start with GENESIS-I.md. Apache-2.0. Collection: Xyntetik Research: GENESIS.
    research artifact
  • The GENESIS existence record
    Research evidence, not a Runner model and no weights: the preregistrations with every amendment, the reports, the experiment code and every counted run of GE-001 to GE-007, failures included. GE-002 failed its preregistered pass on both limbs, a hand-written homeostat outliving the learning subject, and GE-003 was inconclusive. Same collection.
    research record
  • Datasets and collections
    Since 2026-09-15 every weightless report is a Hugging Face Dataset (the ones above, plus the Qwen3-4B tool-use training record), every model card opens with a release or research-artifact status block, and seven collections group the account: Runner Releases, Model Surgery and Scale Recovery, Pruning and Quantization Frontiers, Runner Compatibility Reports, LoRA Training on Quantized Weights, Kvist: distilled agents and GENESIS.

External reproductions and third-party work

What people outside the project have checked.

Adapter byte-equality on a Tesla T4

A Hugging Face community tester reproduced the byte-identical adapter result independently on a Tesla T4 and is credited in the documentation as the first external reproduction.

A 36-prompt tool-choice boundary bank

The same tester published a reusable 36-prompt bank (MIT or Apache-2.0) probing where adapters trained through BF16, Q8_0 and Q4_K_M choose different tools. Run in full through Runner's unlabeled lane: 3 of 36 prompts split, and the Q4-trained adapter carries the widest margins on that bank. Method and scoping: docs/tool-choice-boundary-lane.md.

A community adapter, loaded back

A third-party F16 adapter in llama.cpp's convention loaded and served in Runner, and in doing so found and fixed the F32-only loader gap. Interop is measured in both directions.

What the extra precision buys

Measured against each model's own implementation, not against another engine.

Runner does the model's arithmetic without the shortcuts faster engines take: activations stay in 32-bit float instead of being rounded to 8 bits before every multiply, and attention accumulates in 32-bit float instead of 16. That costs speed where prefill is compute-bound. To see what it buys, the yardstick is the model publisher's own reference implementation run in full precision, not a second engine that may have taken the same shortcuts.

Twelve families against their publishers' implementations in float32, both tiers, 2,000 corpus positions each, measured 2026-09-07
familyweightsRunner, margin-qualified tokens rightother engine, margin-qualified tokens rightofRunner mean KLother engine mean KLdifference real?
SmolLM2 135Mbf161310131013100.00070.0037no
SmolLM2 135MQ8_01310130913100.00800.0164no
Qwen2.5 0.5Bbf161342134213420.00030.0015no
Qwen2.5 0.5BQ4_K_M1328132613420.04740.0518no
Qwen3 0.6Bbf161387138713870.00010.0007no
Qwen3 0.6BQ4_01128112313870.30410.3061no
Qwen3.5 0.8Bbf161356135613560.00040.0011no
Qwen3.5 0.8BQ4_K_M1325132313560.06360.0702no
StableLM 2 1.6Bbf161335133513350.00020.0009no
Granite 4.0-h microbf161437143714370.00330.0040no
Granite 4.0-h microQ4_K_M1356134614370.09740.1063no
Granite 4.2 3Bbf161429142814290.00060.0012no
Granite 4.2 3BQ4_K_M1304130114290.16380.1707no
Trinity Nanobf161340134113600.04540.0450no
Trinity NanoQ8_01326133013610.06330.0616no
Gemma 3 4Bbf161550155015500.00010.0005no
Gemma 3 4BQ4_K_M1491148215500.06690.0764no
Gemma 4 E2Bbf161610161016100.00140.0019no
Gemma 4 E2BQ4_01118110816100.81340.8016no
Qwen3 4Bbf161516151615160.00010.0004no
Qwen3 4BQ4_K_M1396138815160.13980.1434no
Apertus 8BQ4_K_M1343132113690.05090.0790yes

Runner reproduces the reference distribution more closely, almost everywhere. On the measure that uses every position, Runner is closer on twenty-one of these twenty-two rows, at odds no honest reading calls luck. The single exception is Gemma 4 E2B at four bits.

That advantage rarely changes which token wins. The right-hand columns count how often each engine picked the reference's token where the reference had a clear preference. Runner is ahead or level on twenty of twenty-two rows, but only one of those leads is large enough to be a real difference rather than chance: Apertus 8B at four bits, 1343 tokens against 1321. Both statements are true at once, and together they are the honest claim: closer in distribution, near-identical at the token you actually receive.

What this table cost us to learn. An earlier version of this page reported these comparisons over 100 positions instead of 2,000, and at that size a single token moved the number by more than a percentage point. Nearly every difference it showed was noise, including one that appeared to put the other engine ahead on Granite 4.2 at four bits; measured properly, Runner leads that row. StableLM was the exception and it was a real defect of ours, two of them: the architecture normalises with LayerNorm and Runner was applying RMSNorm, and its pre-tokenizer had no entry so a third of the corpus tokenized differently. Both are fixed, the family now reads 100% here, and a test was added that fails whenever an architecture is claimed without a gate behind it. It found four more.

Method and raw outputs: docs/golden-pass-2026-09-07.md, the gate scripts/gold-logits.py (transformers 5.16, float32, eager attention, CPU). Three faults in that harness were found and fixed before any of these numbers counted; they are described in the report. Two families cannot be measured this way yet and are named there rather than scored. The full speed gap is in the benchmark section above, losing rows included.