DPO (Direct Preference Optimisation)
Model CustomisationChapter 7: Speaking in the School's VoiceTerm 122 of 253
Simpler alternative to RLHF that learns directly from preferred vs rejected answers.
At Sunrise School
Gopal asksIs there something simpler than full RLHF?
Bit answersDPO trains directly on pairs: “this reply is better than that one”. Staff picked 1,000 pairs in a week.
In the real world'Answer A is better than B' training.
Watch the short
Short coming soonSeason 5 · Day 25Follow @haigopala
Short coming soonSeason 5 · Day 25Follow @haigopala.telugu
Short coming soonSeason 5 · Day 25Follow @haigopala.hindi
Quick check
Quick check. Can a small student model keep most of the teacher's quality? Yes or no?
Yes! At a fraction of the cost.
Taught together with
Practise it
Try this term and all 253 others in the free Model Training Arena.
Open the Arena →Follow Bit daily: YouTube English · తెలుగు · हिंदी | Instagram English · తెలుగు · हिंदी