Documentation

What is halo-forge?

halo-forge is an RLVR (Reinforcement Learning from Verifiable Rewards) framework that uses compiler feedback as reward signals for iterative model refinement.

The Problem

The Solution

A compiler provides a perfect reward signal — unambiguous, deterministic feedback about code correctness that cannot be gamed.

Architecture

┌──────────┐    ┌──────────┐    ┌──────────┐    ┌───────────┐
│   Data   │ → │   SFT    │ → │   RAFT   │ → │ Benchmark │
└──────────┘    └──────────┘    └──────────┘    └───────────┘

1. Data — Gather training examples from public datasets or LLM generation
2. SFT — Supervised fine-tuning to establish baseline capability
3. RAFT — Iterative verification loop: generate → verify → filter → train
4. Benchmark — Evaluate with pass@k metrics

Results

Production training on Qwen2.5-Coder-7B with 569 C/C++ prompts:

3x improvement over 6 RAFT cycles.

Quick Navigation

Getting Started

• Quick Start — Get running in 30 minutes
• Toolbox Setup — Build the container environment
• Hardware Notes — Strix Halo configuration

Training Pipeline

• Full Pipeline — Complete training workflow
• Data Generation — Prepare training data
• SFT Training — Supervised fine-tuning
• RAFT Training — Reward-ranked fine-tuning
• Benchmarking — Evaluate with pass@k

Verifiers

• Verifier Overview — Choose your verification strategy
• Compile Verifiers — GCC, Clang, MinGW, MSVC
• Test Verifiers — pytest, unittest
• Custom Verifiers — Build your own

Reference

• Configuration — Complete config reference
• CLI Reference — Command-line interface
• Troubleshooting — Common issues

Background

• Theory & Research — Research foundations
• RAFT vs PPO vs GRPO — Algorithm comparison
• Graduated Rewards — Partial credit system
• Learning Rate Strategies — LR recommendations

Meta

• Changelog — Version history
• Contributing — How to contribute