hAIgopala › AI terms › Rapid Fire (3 terms per short) › Day 11

Vanishing Gradient

Neural Networks & Deep LearningChapter 3: Inside the BrainTerm 51 of 253

In deep networks, gradients shrink so early layers stop learning.

At Sunrise School

Gopal asks

Why did the early layers stop learning in a very deep network?

Bit answers

Vanishing gradient: the error signal fades as it travels back through many layers. ReLU and skip connections fix it.

In the real world

Whisper game: message fades down the line.

Watch the short

Short coming soonSeason 5 · Day 11Follow @haigopala

Taught together with

Batch Size · Dropout

Practise it

Try this term and all 253 others in the free Model Training Arena.

Open the Arena →

Follow Bit daily: YouTube English · తెలుగు · हिंदी  |  Instagram English · తెలుగు · हिंदी