What Changed from V7
The two V7 bugs drove every decision here. The multilingual hallucination had a simple fix — filter training data to less than 5% non-ASCII. The tool-call format mismatch required replacing the entire Slice B source and reformatting everything through Qwen2.5's native template pipeline. We also added a fourth slice: 8K English instruction-following examples as a language anchor, because without it the model's distribution was drifting toward tool-calling-only responses. The learning rate dropped again to 1.0e-4 — we're being progressively gentler to preserve the base model's capabilities while adding new ones.
Four specific changes made this version what it is:
1. Native Qwen2.5 Tool-Call Format
V7 used Hermes-style <tool_call> tags as regular text — the model saw them as arbitrary strings, not structured tokens. In V8, all tool-calling examples are formatted via tokenizer.apply_chat_template(tools=...) to produce the exact token sequence the base model expects. This is the difference between teaching the model a foreign syntax and reinforcing a pattern it already knows.
2. English-Only Filtering
A simple threshold did the trick: any training record with more than 5% non-ASCII characters gets dropped. This eliminated the multilingual garbage tokens that V7 was hallucinating mid-response. No complex language detection, no NLP pipeline — just a character-level filter that runs in seconds.
3. Slice D: English Instruction Anchor
We pulled 8K English instruction-following examples from OpenHermes-2.5 and added them as a new training slice. Without this anchor, the model was collapsing toward tool-calling-only responses — ask it a plain question and it would try to emit a function call. Slice D gives the model a reason to just answer in natural language when that's what the user needs.
4. New Slice B Source
We switched from NousResearch/hermes-function-calling-v1 to glaiveai/glaive-function-calling-v2. The Glaive dataset is cleaner, single-source, and much easier to reformat into native Qwen2.5 tool-call templates. Less data wrangling, fewer edge cases.
Dataset and Training
The full dataset grew to 34,104 training rows and 3,789 validation rows, up from 28,862 / 3,206 in V7. The four slices break down as:
| Slice | Content | Rows |
|---|---|---|
| A | Code generation | 7,118 |
| B | Tool calling (Glaive v2) | 10,789 |
| C | Agentic multi-turn | 9,015 |
| D | English instruction following | 7,182 |
Tool coverage sits at 57.6% of the dataset. Training configuration: learning rate 1.0e-4 (down from 1.5e-4), sequence length reduced to 4,096 tokens (from 8,192) to fix A100 OOM crashes, with batch_size=1 and gradient_accumulation_steps=16 for an effective batch size of 16.
Results
Eval loss landed at 0.4837 — slightly higher than V6's 0.4724, but that comparison is misleading. V6 was code-only. V8 covers code generation, tool calling, agentic behavior, and instruction following in one adapter. The loss is higher because the task is harder, and that's fine.
The New Bug: Tag Emission
V8 introduced a subtler problem. The model gets the JSON payload right — correct function name, correct arguments, properly formatted — but sometimes omits the <tool_call> wrapper tags that Ollama's parser needs to route the call. Related issues: trailing garbage text after tool calls, and occasional lowercasing of tool names.
We traced the root cause to quantization. The <tool_call> and </tool_call> tags are multi-token sequences. Q4_K_M quantization loses fidelity on exactly these patterns — the JSON payload is dense and well-learned, but the XML wrapper tags are sparse and get rounded away. The model learned the substance but not the envelope.
What's Next
The tag emission problem is subtle but critical. Two approaches for V9: first, increase tool-calling density in the training data and add a curriculum "polish" phase focused on tag formatting. Second, bump the deployment quantization from Q4_K_M to Q5_K_M for better multi-token pattern fidelity. The extra ~1.5 GB of model size is a small price for reliable tag emission.