Pre-training
Generative AI & LLMsChapter 4: Choosing the LLMTerm 73 of 253
Initial training of an LLM on massive general text to learn language and knowledge.
At Sunrise School
Gopal asksHow did the LLM learn language in the first place?
Bit answersPre-training: reading trillions of words from books and the web and predicting the next word. Months on thousands of GPUs, done by the lab.
In the real worldGoing to school for 12 years.
Watch the short
Short coming soonSeason 2 · Day 16Follow @haigopala
Short coming soonSeason 2 · Day 16Follow @haigopala.telugu
Short coming soonSeason 2 · Day 16Follow @haigopala.hindi
Quick check
Quick check. Does our school do the pre-training? Yes or no?
No! The lab does it once; we build on top.
Practise it
Try this term and all 253 others in the free Model Training Arena.
Open the Arena →Follow Bit daily: YouTube English · తెలుగు · हिंदी | Instagram English · తెలుగు · हिंदी