hAIgopala › AI terms › Rapid Fire (3 terms per short) › Day 25

DPO (Direct Preference Optimisation)

Model CustomisationChapter 7: Speaking in the School's VoiceTerm 122 of 253

Simpler alternative to RLHF that learns directly from preferred vs rejected answers.

At Sunrise School

Gopal asks

Is there something simpler than full RLHF?

Bit answers

DPO trains directly on pairs: “this reply is better than that one”. Staff picked 1,000 pairs in a week.

In the real world

'Answer A is better than B' training.

Watch the short

Short coming soonSeason 5 · Day 25Follow @haigopala

Quick check

Quick check. Can a small student model keep most of the teacher's quality? Yes or no?

Taught together with

GRPO · Distillation

Practise it

Try this term and all 253 others in the free Model Training Arena.

Open the Arena →

Follow Bit daily: YouTube English · తెలుగు · हिंदी  |  Instagram English · తెలుగు · हिंदी