hAIgopala › AI terms › Rapid Fire (3 terms per short) › Day 10

Learning Rate

Neural Networks & Deep LearningChapter 3: Inside the BrainTerm 47 of 253

How big each step is during gradient descent. Too big = overshoot; too small = slow.

At Sunrise School

Gopal asks

How big should each step be?

Bit answers

That's the learning rate. Too big and I overshoot; too small and training takes all week. 0.001 works well for me.

In the real world

Stride length while walking downhill.

Watch the short

Short coming soonSeason 5 · Day 10Follow @haigopala

Taught together with

Activation Function (ReLU, Sigmoid, Softmax) · Epoch

Practise it

Try this term and all 253 others in the free Model Training Arena.

Open the Arena →

Follow Bit daily: YouTube English · తెలుగు · हिंदी  |  Instagram English · తెలుగు · हिंदी