intermediate30 min
Neural Networks
Understand layers, activation functions, and build a simple neural network
What is a Neural Network?
A neural network is a computing system inspired by the brain. It consists of interconnected neurons (nodes) organized in layers. Each connection has a weight that the network adjusts during training.
Network Architecture
Input Layer → Hidden Layer(s) → Output Layer
(data) (computation) (prediction)
Each neuron:
- Receives inputs from the previous layer
- Computes a weighted sum
- Applies an activation function (adds non-linearity)
- Passes the result to the next layer
Key Activation Functions
| Function | Range | Use Case |
|---|---|---|
| ReLU | [0, ∞) | Hidden layers (most common) |
| Sigmoid | (0, 1) | Binary classification output |
| Softmax | (0, 1) sum=1 | Multi-class output |
| Tanh | (-1, 1) | Hidden layers (less common now) |
Building a Neural Network with Keras
Output
Run your code to see output here
How Training Works: Gradient Descent
- Forward pass — Input flows through the network, producing a prediction
- Loss calculation — Compare prediction to the true answer
- Backward pass (backpropagation) — Compute how much each weight contributed to the error
- Weight update — Adjust weights slightly to reduce error
- Repeat thousands of times until the network converges
Key Concepts
- Epoch: One complete pass through the training data
- Batch size: Number of samples processed before updating weights
- Dropout: Randomly deactivates neurons during training — prevents overfitting
- Learning rate: How big the weight updates are (too high = unstable, too low = slow)
Check Your Understanding
What is the primary purpose of an activation function in a neural network?
Next Steps
Neural networks are the foundation of deep learning — with more layers, specialized architectures (CNNs for images, RNNs/Transformers for text), and larger datasets, you can tackle problems like image recognition, language translation, and game playing.