hAIgopala › AI terms › How Bit Really Learns › Day 15

Attention / Self-Attention

Generative AI & LLMsChapter 4: Choosing the LLMTerm 69 of 253

Lets the model weigh how relevant each word is to every other word in the input.

At Sunrise School

Gopal asks

“Can I change my son's bus, he is moving to Kondapur.” How does Bit link “he” to “son”?

Bit answers

Attention lets every word look at every other word. “he” attends strongly to “son”, and “bus” to “Kondapur”.

In the real world

'Bank' relates to 'river' in 'river bank'.

Watch the short

Short coming soonSeason 2 · Day 15Follow @haigopala

Quick check

Quick check. Does attention look only at the word right before? Yes or no?

Practise it

Try this term and all 253 others in the free Model Training Arena.

Open the Arena →

Follow Bit daily: YouTube English · తెలుగు · हिंदी  |  Instagram English · తెలుగు · हिंदी