hAIgopala › AI terms › Rapid Fire (3 terms per short) › Day 15

Mixture of Experts (MoE)

Generative AI & LLMsChapter 4: Choosing the LLMTerm 72 of 253

Model made of many sub-networks ('experts'); only a few activate per token, saving compute.

At Sunrise School

Gopal asks

How do huge models stay fast?

Bit answers

Mixture of experts: a router sends each token to just a few expert sub-networks, like a receptionist sending you to the right desk.

In the real world

Hospital routes you to the right specialist.

Watch the short

Short coming soonSeason 5 · Day 15Follow @haigopala

Taught together with

Encoder vs Decoder · Post-training / Alignment

Practise it

Try this term and all 253 others in the free Model Training Arena.

Open the Arena →

Follow Bit daily: YouTube English · తెలుగు · हिंदी  |  Instagram English · తెలుగు · हिंदी