hAIgopala › AI terms › How Bit Really Learns › Day 16

Pre-training

Generative AI & LLMsChapter 4: Choosing the LLMTerm 73 of 253

Initial training of an LLM on massive general text to learn language and knowledge.

At Sunrise School

Gopal asks

How did the LLM learn language in the first place?

Bit answers

Pre-training: reading trillions of words from books and the web and predicting the next word. Months on thousands of GPUs, done by the lab.

In the real world

Going to school for 12 years.

Watch the short

Short coming soonSeason 2 · Day 16Follow @haigopala

Quick check

Quick check. Does our school do the pre-training? Yes or no?

Practise it

Try this term and all 253 others in the free Model Training Arena.

Open the Arena →

Follow Bit daily: YouTube English · తెలుగు · हिंदी  |  Instagram English · తెలుగు · हिंदी