Mixture of Experts (MoE)
Generative AI & LLMsChapter 4: Choosing the LLMTerm 72 of 253
Model made of many sub-networks ('experts'); only a few activate per token, saving compute.
At Sunrise School
Gopal asksHow do huge models stay fast?
Bit answersMixture of experts: a router sends each token to just a few expert sub-networks, like a receptionist sending you to the right desk.
In the real worldHospital routes you to the right specialist.
Watch the short
Short coming soonSeason 5 · Day 15Follow @haigopala
Short coming soonSeason 5 · Day 15Follow @haigopala.telugu
Short coming soonSeason 5 · Day 15Follow @haigopala.hindi
Taught together with
Practise it
Try this term and all 253 others in the free Model Training Arena.
Open the Arena →Follow Bit daily: YouTube English · తెలుగు · हिंदी | Instagram English · తెలుగు · हिंदी