Documentation

What is halo-forge?

halo-forge is an RLVR (Reinforcement Learning from Verifiable Rewards) framework that uses compiler feedback as reward signals for iterative model refinement.

The Problem

The Approach

A compiler provides deterministic feedback — objective, reproducible results about code correctness.

Architecture

┌──────────┐    ┌──────────┐    ┌──────────┐    ┌───────────┐
│   Data   │ → │   SFT    │ → │   RAFT   │ → │ Benchmark │
└──────────┘    └──────────┘    └──────────┘    └───────────┘

1. Data — Gather training examples from public datasets or LLM generation
2. SFT — Supervised fine-tuning to establish baseline capability
3. RAFT — Iterative verification loop: generate → verify → filter → train
4. Benchmark — Evaluate with pass@k metrics

Example Results

Results from test runs on Qwen2.5-Coder-7B with 569 C/C++ prompts. Your results will vary based on model, dataset, hardware, and configuration.

Improvements were observed over 6 RAFT cycles in our testing.

Quick Navigation

Getting Started

• Quick Start — Get running in 30 minutes
• Toolbox Setup — Build the container environment
• Hardware Notes — Strix Halo configuration

Training Pipeline

• How to Train — Complete step-by-step guide (start here!)
• Full Pipeline — Complete training workflow
• Data Generation — Prepare training data
• SFT Training — Supervised fine-tuning
• RAFT Training — Reward-ranked fine-tuning
• Benchmarking — Evaluate with pass@k

Verifiers

• Verifier Overview — Choose your verification strategy
• Compile Verifiers — GCC, Clang, MinGW, MSVC
• Test Verifiers — pytest, unittest
• Custom Verifiers — Build your own

Reference

• Configuration — Complete config reference
• CLI Reference — Command-line interface
• Troubleshooting — Common issues

Background

• Theory & Research — Research foundations
• Graduated Rewards — Partial credit system
• Learning Rate Strategies — LR recommendations

Meta

• Changelog — Version history
• Contributing — How to contribute