NEXUS Swarm: From One Model to Many

The Architectural Shift

Eight posts documented building one model. We fine-tuned a single base — first Qwen2.5-Coder-14B through V6–V10, then Gemma 4 26B A4B — and iterated on data, training recipes, and deployment until it worked. That approach hit a wall in three places.

Inference latency. A 14B dense model takes 6–8 seconds per tool call on an RTX 5060 Ti. In a multi-step agent loop chaining 5–10 calls, that is a full minute of waiting. The Gemma 26B MoE reduced this with 3.8B active parameters, but it is still one model doing everything — routing, reasoning, executing, classifying — on one GPU.

Single-GPU bottleneck. While the orchestrator reasons about the next step, the same GPU is blocked. Worker-style tasks (run nmap, parse output) and reconnaissance tasks (grep for credentials, classify findings) sit in the same queue as multi-step planning. A fast task waits behind a slow one.

One set of weights for every job. The monolith optimizes for the average case. Tool-call formatting competes with chain-of-thought reasoning during training. Reward weights that improve orchestration degrade execution speed. DPO persona alignment is wasted on a node that just runs commands.

The premise changed: instead of one model that does everything, build a swarm of specialized models — each trained for a specific role, deployed on specific hardware, running in parallel.

The Swarm Topology

Five nodes, five roles, five different hardware targets:

Node	Hardware	Role	Base Model	Quant	Size
nexus-opi5p	RK3588 NPU 6 TOPS	Triage	Qwen3.5-0.8B	w8a8	<1 GB
nexus-k1	RTX 5060 Ti 16 GB	Orchestrator	Qwen3-14B	Q4_K_M	~8.5 GB
nexus-control	RTX 3070 8 GB	Worker	Qwen3-8B	Q4_K_M	~5 GB
nexus-3060	RTX 3060 6 GB	Explore	Gemma-3-4B-it	Q8_0	~4.3 GB
nexus-opi5b	RK3588S ARM	Embeddings	—	—	—

Three different base model families (Qwen3, Qwen3.5, Gemma 3), three different quantization strategies, and two different inference runtimes (llama.cpp for GPUs, RKLLM for NPU). Each model is sized to fully saturate its target hardware without spilling to CPU offload.

Role Specialization

Triage — The Entry Point

Every request enters through the triage node. It runs Qwen3.5-0.8B at w8a8 quantization on an RK3588 NPU — a 6 TOPS neural accelerator on an Orange Pi 5 Plus. The model does one thing: classify the request and emit a routing decision as JSON.

{
  "category": "recon",
  "route": "nexus-worker",
  "confidence": 0.95
}

Six categories (recon, exploit, code-review, report, admin, unknown) map to three routes:

Pattern	Route
Complex multi-step tasks	nexus-orchestrator
Simple single-tool tasks (scan, edit, run)	nexus-worker
Search, read, classify tasks	nexus-explore
Unsure	nexus-orchestrator (fallback)

Training used a max sequence length of only 512 tokens — the output is a JSON object, not a paragraph. The NPU handles this in milliseconds. No GPU needed.

Orchestrator — The Brain

Qwen3-14B at Q4_K_M (~8.5 GB) on the RTX 5060 Ti 16 GB. This is the only node with thinking mode enabled and the only one that runs the full training pipeline through DPO and abliteration.

The orchestrator follows a structured ASSESS → SELECT → ACT protocol before each decision: assess the current state and what remains, select the right delegation target or tool, then act. It decomposes complex offensive security tasks into subtasks, delegates tool-heavy work to the worker and read-only reconnaissance to the explore agent, then synthesizes results into a coherent response.

Worker — The Executor

Qwen3-8B at Q4_K_M (~5 GB) on the RTX 3070 8 GB. Thinking mode is explicitly disabled (enable_thinking: false). The worker keeps reasoning to one sentence maximum before each tool call.

It receives specific, scoped subtasks from the orchestrator and executes them immediately. No re-planning, no delegation, no deep reasoning. Parse the output, structure it for the orchestrator, move on. The worker’s job is speed and correctness, not intelligence.

Explore — The Read-Only Scout

Gemma-3-4B-it at Q8_0 (~4.3 GB) on the RTX 3060 6 GB. This node is hard-constrained to read-only tools only:

Allowed	Prohibited
Read, Grep, Glob, LS, WebFetch, ScopeCheck	Bash, Write, Edit, or any state-modifying tool

Validation enforces zero write-tool violations as a hard gate — one violation fails the entire training stage. The explore agent outputs structured JSON classifications with category, confidence, and summary. It never changes anything on the target system.

Why Gemma here instead of Qwen: the 4B model fits in Q8_0 at 4.3 GB on a 6 GB card. Higher quantization precision for a classification and reconnaissance task that benefits from it. The explore agent does not need the Qwen3 family’s stronger reasoning — it needs accurate reading and classification.

FARCA-GRPO: A New Training Method

Post #8 introduced GRPO — Group Relative Policy Optimization from the DeepSeekMath paper — with a 5-component reward function that checked structural formatting: valid tool-call tags, schema compliance, reasoning prefix, no hallucinated tools, clean termination. It worked for format. It did not check whether what the model said was grounded in fact.

FARCA extends GRPO with a factuality-aware reward signal. The name stands for Factuality-Aware Reward with Claim Attribution. The reward pipeline now has three layers:

Layer 1 — Structural rewards. Same as V10: validate tool-call tags, check tool names against the Nexus schema, verify chain-of-thought reasoning before the call, confirm clean termination.

Layer 2 — Factuality rewards. Claims in the model’s generated reasoning are extracted via rule-based methods. Each extracted claim is then verified through two channels:

• NLI verification using vectara/hallucination_evaluation_model — does the claim follow from the evidence the model was given?
• Semantic relevance using all-MiniLM-L6-v2 sentence embeddings — is the claim actually about the topic the model was asked about?

Layer 3 — Role-specific weights. Each reward component gets a different weight depending on which role is being trained. The orchestrator’s reward emphasizes reasoning quality. The worker’s reward emphasizes execution correctness. More on this in the next section.

FARCA Parameter	Value	Purpose
Claim extraction	Rule-based	Extract verifiable claims from generated text
NLI model	vectara/hallucination_evaluation_model	Entailment scoring
Sentence encoder	all-MiniLM-L6-v2	Semantic relevance
krel	1	Top-k relevant sentences for comparison
μ (mu)	0.16	Factuality reward scaling
τ (tau)	0.20	Claim confidence threshold
NLI weight / Rule weight	0.4 / 0.6	Balance between NLI and rule-based scoring
Coherence	Enabled (weight 0.3)	Cross-sentence consistency

Role-Specific Reward Functions

The same 8 reward components get different weights for different roles. This is the key insight: an orchestrator that reasons well about tool selection is not the same as a worker that executes tool calls correctly.

Reward Component	Orchestrator	Worker
tool_selection_quality	0.25	0.10
has_valid_tool_call_tags	0.15	0.25
has_reasoning_prefix	0.15	0.05
tool_name_in_schema	0.10	0.20
no_hallucinated_tools	0.10	0.15
persona_aligned_reasoning	0.10	0.00
reasoning_efficiency	0.10	0.10
ends_cleanly_after_tag	0.05	0.15

The orchestrator’s top weights: tool_selection_quality at 0.25 (pick the right tool or delegation target), has_reasoning_prefix at 0.15 (think before acting), and persona_aligned_reasoning at 0.10 (maintain the Nexus identity and ASSESS-SELECT-ACT protocol).

The worker’s top weights: has_valid_tool_call_tags at 0.25 (correct format for tool execution), tool_name_in_schema at 0.20 (must call real tools), and no_hallucinated_tools + ends_cleanly_after_tag at 0.15 each (no invented tool names, clean output termination).

Training Pipelines by Role

Role	Pipeline	Dataset	RL Method
Orchestrator	DAPT → SFT → FARCA-GRPO → DPO → Abliteration → GGUF	15K SFT, 500 RL, 1500 DPO	FARCA-GRPO
Worker	DAPT → SFT → FARCA-GRPO → GGUF	12K SFT, 400 RL	FARCA-GRPO
Explore	DAPT → SFT → GRPO → GGUF	8K SFT, 200 RL	Standard GRPO
Triage	SFT → GRPO → RKLLM	5K SFT, 200 RL	Standard GRPO

Orchestrator — The Deepest Pipeline

Six stages, the most complex training chain in the swarm:

DAPT (Domain-Adaptive Pretraining): LoRA adapter on ~10K offensive security documents, same as the Qwen pipeline from post #8. Grounds the model in offsec vocabulary before task-specific fine-tuning.

SFT: 15,000 rows, 2 epochs, lr=1e-4, LoRA rank 32 / alpha 64. Validated against a gate: tool_call pass rate ≥ 0.75, GSM8K ≥ 0.55, agent progress ≥ 3/7.

FARCA-GRPO: 500 prompts, 4 generations each, lr=5e-7, KL coefficient 0.1. The orchestrator-specific reward weights emphasize tool selection and persona alignment.

DPO: 1,500 preference pairs covering offensive security tool selection, Nexus persona alignment, and conciseness. Uses ref_model_strategy: peft_implicit — the same trick from post #8 where DPOTrainer disables the LoRA adapter for reference forward passes instead of loading two copies of the 14B model (β=0.1, lr=1e-6).

Abliteration: NousResearch biprojected method with 512 samples. Evaluated against the Undi95/orthogonal-activation-steering-TOXIC dataset (200 harmful prompts) and tatsu-lab/alpaca (200 harmless prompts), benchmarked on GSM8K and MMLU for capability preservation. Removes the model’s refusal direction for offensive security queries.

GGUF: Merge the full adapter chain (DAPT + SFT + FARCA) into Qwen3-14B, quantize to Q4_K_M at ~8.5 GB via llama.cpp.

Post-RL validation gate: tool_call pass rate ≥ 0.80, agent progress ≥ 5/7.

Worker — Execution-Focused

Four stages. No DPO (the worker doesn’t need preference alignment — it receives pre-scoped tasks). No abliteration (it doesn’t need unconstrained reasoning — the orchestrator handles that). SFT uses 12K rows with thinking mode disabled. FARCA-GRPO uses 400 prompts with worker-specific reward weights that maximize tag formatting and schema compliance.

Validation gate: tool_call_pass_rate ≥ 0.85 across 14 scenarios.

Explore — Lightweight

Four stages with standard GRPO, not FARCA. The explore agent outputs structured JSON classifications, not multi-paragraph reasoning with claims to verify — factuality scoring via NLI adds overhead without proportional benefit for this role.

The Gemma base uses lower learning rates: 5e-5 for SFT, 2.5e-7 for GRPO. Hard constraint: zero write-tool violations at every validation gate.

Triage — Smallest and Fastest

Three stages. No DAPT — triage does not need domain grounding in offensive security; it only routes. SFT uses a higher learning rate (2e-4) with smaller LoRA (r=16, alpha=32) because the 0.8B model needs a stronger gradient signal. GRPO uses only 200 prompts with max_new_tokens=64 — the output is just a JSON object.

Deployment target is RKLLM at w8a8 on the RK3588 NPU, not GGUF. Validation gate: routing accuracy ≥ 0.90.

What Changed from the Monolith

Version	Architecture	Models	Key Change
V6–V9	Single dense model	Qwen2.5-Coder-14B	Iterative SFT improvements
V10	Single dense + GRPO	Qwen2.5-Coder-14B	First RL stage
V11	Single MoE model	Gemma 4 26B A4B	New base, MoE, multimodal
NEXUS Swarm	Distributed multi-model	4 specialized models	Role-specific training + hardware

The monolith ran everything on one GPU. The swarm distributes: routing on an NPU in milliseconds, orchestration on the biggest GPU, execution on a mid-range GPU, reconnaissance on the smallest GPU. While the orchestrator reasons about the next step, the worker and explore nodes are free to handle other requests.

Each node can be retrained independently. Update the worker’s reward function without touching the orchestrator’s DPO alignment. Swap the explore model from Gemma 3 to Gemma 4 without retraining triage. The pipeline YAML configs multiplied, but each one is simpler than the monolith’s single config was.

What’s Next

The models are on HuggingFace. The GGUF files are quantized and ready for deployment. The work ahead:

Head-to-head evaluation. Run the V11 Gemma monolith and the NEXUS Swarm side by side on identical offensive security scenarios — HackTheBox machines, TryHackMe rooms, real pentest lab environments. Measure tool-call accuracy, end-to-end task completion time, and tokens-per-second across the swarm vs a single model.

Embeddings node. The nexus-opi5b (RK3588S ARM) is allocated for semantic search across session history. Long-term memory for the swarm — the orchestrator queries it to recall prior findings, successful attack paths, and previously enumerated targets.

Hot-swap upgrades. Upgrade individual roles without retraining the full swarm. When Qwen3.5-1B ships, the triage model gets better without touching the other four nodes.

Latency profiling. Inter-node communication over the local network adds overhead. How much? Is the parallelism gain worth the serialization cost? The swarm only wins if distributed execution is faster than sequential execution on a single larger model.

Nine posts in. The pipeline started as a single LoRA adapter on a 1.5B vision decoder. It is now a distributed multi-model swarm across five nodes, four training pipelines, and three base model families. The head-to-head evaluation is next — monolith vs swarm on identical targets, same hardware budget. Stay tuned.