Build logs and technical notes from the C3S engineering team.
A frozen Swin-B encoder, an MLP projector, and a LoRA adapter walk into a training run. Syntax validity jumped from 19.5% to 61% — the model learned to emit code-shaped output, not transcribe it.
Three LoRA configs on a 14B base model. The aggressive winner (r=64, lr=3e-4) hit eval_loss 0.4724 and actually improved GSM8K math scores by 12%. No catastrophic forgetting.
We ran the winning config for 3 epochs on 8K rows instead of 1 epoch on 26K. Same compute budget, worse result. The lesson: more unique examples beats more passes.
Merging the LoRA adapter into the base model and quantizing to Q4_K_M and Q5_K_M for local serving. The model runs on consumer hardware via llama.cpp and Ollama.
V6 was code-only: zero tool-calling, zero multi-turn reasoning. V7 blended three capability slices to restore what we'd trained away — but introduced multilingual hallucination and format mismatches.
Switched to Qwen2.5's native tool-call template, filtered out non-English garbage, and added an instruction-following anchor slice. The multilingual hallucination died. The tag emission didn't.
Two-phase curriculum training: 80% full dataset, 20% tool-calling polish at higher LR. Synthetic multi-tool examples, strict tag validation, and a bump to Q5_K_M for deployment fidelity.