Activation Function (ReLU, Sigmoid, Softmax)
Neural Networks & Deep LearningChapter 3: Inside the BrainTerm 43 of 253
Adds non-linearity so networks can learn complex patterns. Softmax turns scores into probabilities.
At Sunrise School
Gopal asksHow does a neuron decide to fire?
Bit answersAn activation function. ReLU keeps positive signals and drops negatives; softmax turns final scores into probabilities like Fees 87%.
In the real worldReLU: if negative → 0, else keep.
Watch the short
Short coming soonSeason 5 · Day 10Follow @haigopala
Short coming soonSeason 5 · Day 10Follow @haigopala.telugu
Short coming soonSeason 5 · Day 10Follow @haigopala.hindi
Quick check
Quick check. Is a bigger learning rate always faster to a good model? Yes or no?
No! Too big, and it jumps past the best point.
Taught together with
Practise it
Try this term and all 253 others in the free Model Training Arena.
Open the Arena →Follow Bit daily: YouTube English · తెలుగు · हिंदी | Instagram English · తెలుగు · हिंदी