AI Glossary
What is Direct Preference Optimization?
Overview
Direct Preference Optimization is a training method that tunes language models from preference data without a separate reinforcement learning loop. It matters because many labs and open-model teams use DPO-style methods to align responses, improve instruction following, and make models cheaper to refine after supervised training.
Why it matters
DPO is a common post-training technique behind instruction-tuned and preference-aligned models.
Where it appears in AI research
- Open-weight model training reports
- Alignment and post-training papers
- RLHF alternative discussions
- Model release technical notes