# Reinforcement Fine-Tuning

> Reinforcement Fine-Tuning (RFT) trains AI models using verifiable rewards on tasks with objectively correct answers, producing emergent reasoning capabilities.

Source: https://metavert.io/reinforcement-fine-tuning  
Published: 2026-03-08  
Updated: 2026-10-07

**Reinforcement Fine-Tuning (RFT)** is a training paradigm that uses verifiable rewards—objective signals from tasks with checkable correct answers—to teach [language models](https://metavert.io/large-language-models) to reason more effectively. Unlike [RLHF](https://metavert.io/rlhf), which relies on subjective human preferences, RFT exploits the fact that many valuable tasks (math, coding, logic puzzles, scientific reasoning) have verifiably correct solutions.

The approach gained prominence through DeepSeek-R1, which demonstrated something remarkable: when trained purely with reinforcement learning on problems with verifiable answers, a language model spontaneously develops chain-of-thought reasoning, self-correction, and even the ability to allocate more compute to harder problems. No one explicitly taught the model to "think step by step"—it emerged from the optimization pressure of getting correct answers.

This emergence of reasoning from pure RL was one of the most significant findings in AI development circa 2024-2025. It suggested that complex cognitive behaviors like planning, reflection, and error correction aren't things that need to be explicitly programmed or demonstrated through supervised examples. Given the right reward signal and enough optimization, they arise naturally—a finding with profound implications for how we think about intelligence.

RFT is particularly powerful for creating specialized [reasoning models](https://metavert.io/reasoning-models). By fine-tuning on domain-specific verifiable tasks—mathematical proofs, code that must pass test suites, scientific predictions with known outcomes—models can develop deep expertise in specific reasoning domains. OpenAI offers RFT as a fine-tuning option, and the technique underlies much of the [benchmark](https://metavert.io/ai-benchmarks) improvement seen in reasoning-focused models. Combined with [DPO](https://metavert.io/direct-preference-optimization) for style alignment, RFT represents a powerful two-pronged approach to building both capable and well-behaved AI systems.

### 2026 developments: GRPO in open tooling

The algorithm behind most open RFT is [GRPO](https://metavert.io/grpo) (Group Relative Policy Optimization), introduced in the DeepSeekMath paper in February 2024 as a variant of PPO. It samples a group of answers to each prompt and rewards each relative to the others, so it needs neither a separate value model nor a learned reward model. By 2026 this had moved from research code into general fine-tuning tools. Unsloth's documentation, for example, describes GRPO training with custom reward functions, supports variants selected through a loss-type setting, states that 5GB of VRAM is enough for models of 1.5B parameters or fewer, and advises waiting at least 300 steps before expecting the reward to rise. Hosted RFT remains narrower: OpenAI's guide, as accessed in October 2026, supports it on o-series reasoning models and currently only o4-mini, and notes that RFT cannot help where the base model's success rate is 0% or where scores are already at the maximum. In practice RFT is the second stage of a [post-training](https://metavert.io/post-training) recipe, after supervised fine-tuning has brought the model to the point of occasional success; the two are contrasted in [supervised vs reinforcement fine-tuning](https://metavert.io/compare/supervised-fine-tuning-vs-reinforcement-fine-tuning).

### From answer checks to environments

Early RFT rewarded a single final answer. Agentic RFT rewards the outcome of a multi-step episode inside an [RL environment](https://metavert.io/rl-environments): a task, a sandbox with tools, and a verifier. Open libraries such as Prime Intellect's `verifiers` (MIT license) package these as reusable units that plug into a trainer. Research in 2026 has focused on generating them: Envs-FORGE (August 2026) reports that 100 verified synthetic terminal environments, tuned toward the edge of the model's ability, lifted Qwen 3.5 models from 4B to 35B by 6.8 to 9.2 points on its tb-core benchmark, and EnvFactory (May 2026) reports gains of up to 15% on BFCLv3 from 85 verified tool environments. Both are author-reported.

### Rubric judges for tasks without a checker

The original definition of RFT assumes a verifiably correct answer. A growing share of 2026 work relaxes that by using a rubric-scored [LLM judge](https://metavert.io/llm-as-a-judge) as the reward. OpenAI's RFT graders include a `score_model` type in which another model returns a score, alongside string checks and weighted combinations. Kim and Ahmad (alphaXiv, May 2026) trained a 4B model with GRPO on an evidence-selection task using rubric-based judges, on the grounds that several answers can be valid and overlap metrics punish that; the training-set rubric score rose from 0.3 to 0.6. Alibaba's Qwen-AgentWorld (June 2026) describes "hybrid rubric-and-rule rewards". Judge-based rewards widen what RFT can train, but they also import the judge's blind spots: a policy optimised against a model-written score will find whatever that score fails to penalise, which is the [reward hacking](https://metavert.io/reward-hacking) problem in a new form. Holding out a separate judge or human review for evaluation is the usual safeguard.

## Related Topics

- [RLHF](https://metavert.io/rlhf)
- [Reasoning Models](https://metavert.io/reasoning-models)
- [Test-Time Compute](https://metavert.io/test-time-compute)
- [Reinforcement Learning](https://metavert.io/reinforcement-learning)
- [AI Model Training](https://metavert.io/ai-model-training)
- [Direct Preference Optimization](https://metavert.io/direct-preference-optimization)
- [Open-Weight Models](https://metavert.io/open-weight-models)
- [GRPO](https://metavert.io/grpo) — the algorithm behind most open RFT
- [RL Environments](https://metavert.io/rl-environments) — tasks, sandboxes and verifiers that supply the reward
- [Post-Training](https://metavert.io/post-training) — the pipeline stage RFT belongs to
- [LLM-as-a-Judge](https://metavert.io/llm-as-a-judge) — rubric scoring as a reward signal
- [Reward Hacking](https://metavert.io/reward-hacking) — the main failure mode of grader-driven training
- [Supervised vs Reinforcement Fine-Tuning](https://metavert.io/compare/supervised-fine-tuning-vs-reinforcement-fine-tuning) — demonstrations versus graders

## Further Reading

- [The State of AI Agents in 2026](https://meditations.metavert.io/p/the-state-of-ai-agents-in-2026)
- [DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning](https://arxiv.org/abs/2501.12948) — arXiv, January 2025
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models (introduces GRPO)](https://arxiv.org/abs/2402.03300) — arXiv, February 2024
- [Reinforcement fine-tuning guide](https://developers.openai.com/api/docs/guides/reinforcement-fine-tuning) — OpenAI developer documentation, accessed October 2026
- [Reinforcement Learning (RL) Guide](https://unsloth.ai/docs/get-started/reinforcement-learning-rl-guide) — Unsloth documentation, accessed October 2026
- [verifiers: environments for training and evaluating LLMs](https://github.com/PrimeIntellect-ai/verifiers) — Prime Intellect, GitHub
- [Envs-FORGE: Frontier-Optimized Reward-Grounded Environment Synthesis for Agent RL (Wu et al.)](https://arxiv.org/abs/2608.14312) — arXiv, August 2026
- [EnvFactory: Scaling Tool-Use Agents via Executable Environments Synthesis and Robust RL (Xu et al.)](https://arxiv.org/abs/2605.18703) — arXiv, May 2026
- [Reinforcing Recursive Language Models (Kim and Ahmad)](https://www.alphaxiv.org/blog/reinforcement-learning-for-rlms) — alphaXiv, May 2026
- [Qwen-AgentWorld: Language World Models for General Agents](https://arxiv.org/abs/2606.24597) — arXiv, June 2026