Fine-Tuning

Fine-tuning is the process of further training a pre-trained foundation model on a specialized dataset to adapt its capabilities for specific tasks, domains, or behavioral characteristics—without the cost of training from scratch.

Fine-tuning has become a critical capability in the AI ecosystem. A general-purpose language model trained on broad internet data can be fine-tuned on medical literature to become a clinical assistant, on legal documents to become a legal analyst, or on a company's internal knowledge base to become a domain-specific expert. The technique requires orders of magnitude less data and compute than pre-training—typically thousands to millions of examples versus the trillions of tokens used in pre-training.

Parameter-efficient fine-tuning methods have dramatically reduced costs. LoRA (Low-Rank Adaptation) and QLoRA modify only a small fraction of model parameters, enabling fine-tuning of billion-parameter models on consumer GPUs. This democratization means that individual developers and small companies can create specialized AI systems tailored to their needs—a key enabler of the Creator Era in AI.

The distinction between fine-tuning and other adaptation techniques is increasingly blurred. Retrieval-augmented generation (RAG) provides context without changing model weights. Prompt engineering steers behavior through input design. Reinforcement learning from human feedback (RLHF) aligns models with human preferences. In practice, production AI systems combine multiple techniques: a fine-tuned base model with RAG for current information, system prompts for behavioral guardrails, and agentic tool use for real-world capability.

Supervised fine-tuning

The default form is supervised fine-tuning (SFT): the model is shown input–output pairs and trained to reproduce the outputs, using the same next-token objective as pre-training. SFT is the right tool when good demonstrations exist—transcripts in a house style, labelled extractions, tool-call traces from a stronger model. Its ceiling is the data: the model learns to imitate the examples, including their mistakes, and cannot exceed them. Full fine-tuning updates every weight, which gives the most capacity to change the model but produces a complete new copy per task and, on small datasets, a greater risk of the model losing general abilities it had before (see continual learning).

Parameter-efficient fine-tuning: adapters, LoRA and QLoRA

Parameter-efficient fine-tuning (PEFT) freezes the base model and trains a small number of added parameters. Adapter modules (Houlsby et al., 2019) came within 0.4% of full fine-tuning on the GLUE benchmark while adding 3.6% parameters per task. LoRA (Hu et al., 2021) instead injects trainable low-rank matrices into each Transformer layer; against GPT-3 175B fine-tuned with Adam, the authors report 10,000 times fewer trainable parameters and a threefold reduction in GPU memory. QLoRA (Dettmers et al., 2023) backpropagates through a frozen base model quantized to 4 bits into LoRA adapters, which was enough to fine-tune a 65B-parameter model on a single 48GB GPU while, in the authors' tests, preserving full 16-bit fine-tuning task performance. The trade-offs against updating all weights are covered in LoRA vs full fine-tuning.

Reinforcement fine-tuning

Reinforcement fine-tuning replaces the target outputs with a grader. The model generates several attempts, a program or judge model scores them, and an algorithm such as GRPO shifts the weights toward higher-scoring behaviour. It suits tasks where a result is easier to check than to demonstrate: code that must pass tests, answers with a verifiable value, multi-step tool use with a measurable end state. It needs a model that already succeeds some of the time; OpenAI's documentation states that if a model has a 0% success rate on a task, "you cannot bootstrap to higher performance levels" this way. A common recipe is therefore SFT first and RL second, compared directly in supervised vs reinforcement fine-tuning. Both are stages of what model developers call post-training.

When to fine-tune, optimize prompts, or retrieve

The three approaches fix different problems. If the model lacks facts that change or are private, retrieval supplies them without touching weights. If the model can do the task but does it inconsistently, prompt optimization is cheaper to try first: GEPA (July 2025), a reflective prompt optimizer, reported outperforming GRPO by 6% on average across six tasks while using up to 35 times fewer rollouts. That is one paper's result on its chosen tasks, but it is a reasonable argument for exhausting prompt-side gains before training. Fine-tuning earns its cost when the goal is something a prompt cannot buy: a smaller, cheaper or faster model that matches a larger one on a narrow task, a behaviour that must hold without long instructions, or a skill the base model does not show however it is prompted.

Open bases as of October 2026

Fine-tuning requires weights, so in practice it starts from an open-weight model or a vendor's hosted tuning service. Among open bases, Alibaba's Qwen family is a frequent choice: Qwen3.8-27B, released in August 2026 under Apache 2.0, has a 262,144-token native context, and Unsloth's documentation states that QLoRA on it works with 24GB of VRAM while LoRA needs more than 36GB. Its mixed linear-attention architecture is a reminder that adapters are tied to the exact base they were trained on. DeepSeek's V4 series also publishes checkpoints, but at 284B and 1.6T total parameters those are served or distilled far more often than tuned. Hosted reinforcement fine-tuning is narrower: OpenAI's documentation, as accessed in October 2026, lists it for o-series reasoning models and currently only o4-mini.