LoRA training
Train the file you actually ship.
A LoRA adapter is a small set of extra weights that teaches an existing model a task without rewriting the whole model. Runner trains one directly through the quantized GGUF file used for inference, the same 4-bit or 8-bit file you will serve, on the CPU or on a CUDA device, with no separate full-precision copy and no training framework. Same data, same seed, same configuration: the same adapter, byte for byte.
Why this is unusual, and why it matters
Two numerical paths is one too many.
The usual pipeline
Fine-tune through a 16-bit copy in a Python framework. Quantize the result to a 4-bit file. Serve that file in a different engine. Three tools, two numerical paths, and the thing you trained is never exactly the thing you sampled from in production.
What Runner does instead
The serving forward pass is the training forward pass. The policy you sample is the policy you train: given a rollout's token ids, the trainer scores them bit for bit as a CPU server sampled them, with the prompt read one token per forward (-b 1), gated in CI on every family the trainer admits. So the train/infer mismatch that silently breaks on-policy learning does not occur. There is no FP16 copy to keep, and no framework to install.
Adaptation as an auditable artifact
Every adapter is written beside a provenance record: base, data and adapter sha256, seed, full config, binary sha, compiler, OS and architecture. Two runs with the same inputs write the same file, gated in CI by sha256. That is a stronger statement than "repeatable within tolerance", and it is scoped to the same build on purpose.
Measured
0.69 to 1.00, on a file you can download.
Measured 2026-08-22 on Qwen3-4B-Runner-ToolUse-Q4_K_M, a public artifact whose card carries the sidecar and the training command. The claim scope is one base and one task. The 8B and 14B ladder is open work, and these numbers are not a promise about other families until they are measured.
Qwen3-4B Q4_K_M, the frozen 4-bit serving weights
directly through the quantized inference artifact, CPU or CUDA-assisted
0.69 before, 1.00 after
two independent runs, same sha256 on the adapter file
adapters trained through BF16 and Q8_0: cosine 0.9998; through Q4_K_M: 0.9926. A measurably different object, capability-equivalent on this task's supervised decisions; on the gold-completion region they coincide to about 0.0003 nat. Not equivalent everywhere: an external 36-prompt boundary bank finds 3 of 36 prompts where the three adapters choose different tools, and the Q4-trained adapter carries the widest margins on that bank
nll per token 4.063 to 4.026: the adapter leaves unrelated text alone
the adapter scores the same 1.00 served by stock llama.cpp; a community F16 adapter loads back into Runner
--merge-lora folds an adapter into the base to produce a standalone GGUF any runtime can serve. Merging into Q8_0 or F16 keeps the 1.00, verified in stock llama.cpp. Merging into the 4-bit base returns the score to 0.69, the base's own number, because 98.55% of the weight bytes round back to the base's quantization grid. Perplexity still looks fine while it happens. A scale sweep shows survival is monotone in delta magnitude: erased through 2x, partial at 4x, full at 8x. Serve adapters with --lora, the exact form, or merge into 8-bit or better.
Everything the adapter workflow covers
Train, serve, merge, prove. One binary.
Train from your data
Plain text, prompt and completion pairs with the prompt masked from the loss, or exact token ids. Per-example weights, negative ones included, and an opt-in end-of-turn target so a trained answer also learns to stop. Warnings before the first step when small data is likely to be memorized.
Continue and checkpoint
Checkpoints every N steps, written so a failure keeps the previous one. Continue an existing adapter with --lora: the record names the adapter it started from, so the history stays traceable.
Preferences and rewards
DPO from chosen and rejected answers, with the frozen base as the reference and no second model copy. A reinforcement loop that samples, scores and trains in rounds, and checks every round that the trainer scores the rollouts bit for bit as the server sampled them.
Serve one, or many
Serve an adapter beside the frozen base on the CPU, CUDA or Metal, with its scale adjustable. Or load up to 16 adapters and pick one per request by model name, each request isolated from the others' cached prompts.
Merge without losing it
Fold an adapter into a standalone GGUF for any runtime. Every merge measures how much of the fine-tune survived the output's quantization grid, and refuses one that would erase it.
Prove where it came from
Every run writes a record of its base, data, starting adapter, settings and build. Sign records and adapter files, require the signature at load, walk an adapter's lineage back to its base and data, and replay a served answer with the adapter that gave it.
Each feature, with its commands, limits and measurements: the complete adapter guide. Check a model first with --adapt-info; adapters load and save in llama.cpp's format, both ways.
# train through the quantized base you will serve
./runner -m base-Q4_K_M.gguf --train data.jsonl --train-steps 200 \
--lora-rank 8 --train-out adapter.gguf
# adapter.gguf.train.json now records base/data/adapter sha256, seed, config, binary
# serve the adapter beside the frozen base (the exact form)
./runner -m base-Q4_K_M.gguf --lora adapter.gguf --serve
# fold it into a standalone file for any GGUF runtime: 8-bit or better
./runner -m base-Q4_K_M.gguf --lora adapter.gguf \
--merge-lora merged-Q8_0.gguf --quant q8_0
# teacher-forced logprobs for evals and rewards
./runner -m base-Q4_K_M.gguf --lora adapter.gguf -f eval.txt --scoreFlags: --adapt-info, --train, --train-steps, --lr, --train-ctx, --train-out, --save-every, --lora-rank, --train-eot, --train-logprobs, --train-dpo, --dpo-beta, --lora, --lora-scale, --lora-sig, --adapter, --merge-lora, --merge-allow-erased, --score, --lineage. Command-line reference. scripts/train-grpo-lite.py closes the loop into seeded, replayable reinforcement fine-tuning.
Run it
Three commands, one binary.
Serving an adapter runs on the CPU, on CUDA and on Metal: an offloaded block applies the delta from adapter weights held in device memory, and the CPU-versus-GPU gap with the adapter is no wider than without it. Training is hosted on the CPU. The backward's transposed matvecs run on CUDA behind RUNNER_TRAIN_GPU=1, which since its position grid is the faster arm on a consumer card and writes the same adapter bytes either way: Qwen2.5-7B Q4_K_M at 64.8 s/step against 76.5 on 8 CPU threads, RTX 3070, measured 2026-09-07. The forward pass is still CPU-hosted, so CUDA training end to end is open work. Adapters follow llama.cpp's naming convention in both directions: F16 adapters from convert_lora_to_gguf load, and the adapters Runner writes load in llama.cpp.
Where the numbers come from, with design, gates and failure modes: docs/adaptation-engine.md. The reproducibility scope, with what is and is not claimed: receipts and the determinism scope. The external boundary bank and its lane: docs/tool-choice-boundary-lane.md.