hAIgopala › AI terms › Rapid Fire (3 terms per short) › Day 14

Multi-Head Attention

Generative AI & LLMsChapter 4: Choosing the LLMTerm 70 of 253

Several attention mechanisms run in parallel, each focusing on different relationships.

At Sunrise School

Gopal asks

Can attention track several things at once?

Bit answers

Multi-head attention: one head tracks who (the son), another what (a bus change), another where (Kondapur), all in parallel.

In the real world

Several readers each looking for different clues.

Watch the short

Short coming soonSeason 5 · Day 14Follow @haigopala

Taught together with

Max Tokens · Top-p / Top-k Sampling

Practise it

Try this term and all 253 others in the free Model Training Arena.

Open the Arena →

Follow Bit daily: YouTube English · తెలుగు · हिंदी  |  Instagram English · తెలుగు · हिंदी