# Backpropagation Demystified: How Neural Networks Learn
> Learn the fundamentals of neural network training: forward propagation, loss functions, gradient descent, the chain rule, and activation functions like ReLU.

Tags: neural-networks, backpropagation, machine-learning, artificial-intelligence, deep-learning, gradient-descent, data-science
## What is Backpropagation?
* A neural network is an optimization problem to find the best weights and biases to minimize error.
* Backpropagation uses calculus (chain rule) to trace how weights affect final loss.

## Forward Propagation
* Each neuron multiplies input by weight and adds bias: `neuron(x) = wx + b`.
* This pass creates a prediction and intermediate values needed for the backward pass.

## Measuring Mistakes (Loss Function)
* Uses Mean Squared Error (MSE): `Loss = (1/n) Σ (ŷi - yi)²`.
* Squaring ensures negatives don't cancel and larger mistakes are penalized more heavily.

## Gradient Descent
* The gradient is the direction of steepest ascent.
* Training involves moving in the opposite direction (steepest descent) to lower loss.
* Learning rate controls step size.

## The Chain Rule
* To find `∂loss/∂w`, multiply the upstream gradient by the local gradient.
* `∂loss/∂w = (∂loss/∂ŷ) × (∂ŷ/∂w)`
* This allows for efficient reuse of values without redundant computation.

## Backward Propagation Example
* Calculations show updating weights (w) and biases (b) based on gradients and a learning rate (lr = 0.01).
* Real step example: New loss dropped from 3.61 to 2.87 in a single step.

## Activation Functions
* Linear neurons only produce straight lines; non-linearity is needed for complex data.
* ReLU (Rectified Linear Unit) function: `f(x) = max(0, x)` allows the network to fit curves while staying differentiable.

## The Training Loop
* The cycle includes: 1. Forward Pass, 2. Loss Calculation, 3. Backward Pass, 4. Parameter Update.
* This logic scales from 1 neuron to large language models with billions of parameters.
---
This presentation was created with [Bobr AI](https://bobr.ai) — an AI presentation generator.