Experimental Features

This section contains features that are currently in development and testing. These modules extend halo forge beyond code generation training into new domains.

Status: These features are functional but may change significantly as we iterate on the designs.

Current Version: v1.1.0

Available Experimental Modules

Vision-Language Training (Phase 3)

Train vision-language models using RLVR with perception-aware verification.

# Validate configuration first
halo-forge vlm train --model Qwen/Qwen2-VL-7B-Instruct --dataset textvqa --dry-run

# Then run training
halo-forge vlm train --model Qwen/Qwen2-VL-7B-Instruct --dataset textvqa

Full VLM Documentation

Audio-Language Training (Phase 4)

Train speech and audio models using RLVR with ASR/TTS verification.

# List available datasets
halo-forge audio datasets

# Validate configuration
halo-forge audio train --model openai/whisper-small --dataset librispeech --dry-run

# Train with ASR verification
halo-forge audio train --model openai/whisper-small --dataset librispeech --cycles 3

Full Audio Documentation

Reasoning & Math Training (Phase 5)

Train models on mathematical and logical reasoning with symbolic verification.

# List available datasets
halo-forge reasoning datasets

# Train on GSM8K
halo-forge reasoning train --model Qwen/Qwen2.5-3B-Instruct --dataset gsm8k --cycles 3

Full Reasoning Documentation

Agentic / Tool Calling (Phase 6)

Train models to generate structured function calls with schema-aware verification.

Key Features:

• Graduated reward structure (partial credit for valid JSON, schema compliance)
• Hermes format output (XML-style tags in ChatML, compatible with Qwen/Llama)
• Supports multi-turn conversations with tool results

# List available datasets
halo-forge agentic datasets

# Validate configuration
halo-forge agentic train --model Qwen/Qwen2.5-7B-Instruct --dataset xlam --dry-run

# Train on xLAM dataset
halo-forge agentic train --model Qwen/Qwen2.5-7B-Instruct --dataset xlam --cycles 3

Full Agentic Documentation

Inference Optimization (Phase 2)

Optimize trained models for deployment without full retraining.

# Check dependencies and validate config
halo-forge inference optimize --model models/trained --dry-run

# Export to GGUF
halo-forge inference export --model models/trained --format gguf --output model.gguf

Full Inference Documentation

Liquid AI LFM2.5 Models

Liquid AI’s LFM2.5 family is experimentally supported for RAFT training. These models feature a hybrid convolutional architecture optimized for edge deployment.

Quick test:

halo-forge benchmark run \
  --model LiquidAI/LFM2.5-1.2B-Instruct \
  --prompts data/prompts.jsonl \
  --verifier mingw \
  --samples 5

Initial results (5 Windows API prompts, 3 samples each):

• pass@1: 33%
• Compile rate: 40-60%
• PEFT/LoRA: Compatible
• Generation speed: ~47s per batch of 4

Stability Levels

Feedback

These features are actively developed based on testing results. If you encounter issues or have suggestions:

1. Check the Troubleshooting guide
2. Review the Changelog for recent changes
3. Open an issue on GitHub with reproduction steps

Planned Features