Self-supervised Learning
Learning TypesChapter 1: Learning from EnquiriesTerm 24 of 253
Model creates its own labels from the data itself. This is how LLMs are pre-trained.
At Sunrise School
Gopal asksHow did the language brain learn English with no labels at all?
Bit answersSelf-supervised learning: hide a word and predict it. “School reopens on ___” → “Monday”. Repeated billions of times.
In the real worldHide the next word and predict it.
Watch the short
Short coming soonSeason 5 · Day 5Follow @haigopala
Short coming soonSeason 5 · Day 5Follow @haigopala.telugu
Short coming soonSeason 5 · Day 5Follow @haigopala.hindi
Quick check
Quick check. Is self-supervised learning how LLMs are pre-trained? Yes or no?
Yes! Billions of fill-in-the-blank guesses.
Taught together with
Practise it
Try this term and all 253 others in the free Model Training Arena.
Open the Arena →Follow Bit daily: YouTube English · తెలుగు · हिंदी | Instagram English · తెలుగు · हिंदी