Neural Networks flashcards, generated for you.

A neural network learns by adjusting many simple processing units (neurons) to recognize patterns in data. Each neuron takes inputs, weighs their importance, and passes a transformed result forward—mimicking how biological brains process information through connected cells.

Each layer learns increasingly complex features. Early layers detect simple patterns (edges, textures), middle layers combine them (shapes, parts), and deeper layers recognize high-level concepts (objects, faces). This hierarchy lets networks solve complex problems.

An activation function introduces non-linearity, allowing the network to learn curved decision boundaries and complex relationships. Without it, stacking layers would just create one big linear transformation—unable to solve problems like XOR.

The forward pass computes output by propagating input through each layer: multiply inputs by weights, add bias, apply activation function, repeat. Mathematically: a = σ(Wx + b), where W is weight matrix, b is bias, and σ is activation.

Backpropagation efficiently computes how much each weight contributed to the error by working backward from output to input, using the chain rule. This lets us update thousands of weights quickly—it's the algorithm that makes training neural networks practical.

The loss function quantifies how wrong the network's predictions are compared to true answers (e.g., mean squared error for regression, cross-entropy for classification). We minimize it during training to improve predictions.

A weight (W) is a learned parameter that scales the importance of an input. A bias (b) is a learned offset added to the weighted sum before activation: z = Wx + b. Together they allow neurons to learn any linear transformation.

Gradient descent computes the gradient (derivative) of loss with respect to each weight, then updates weights in the opposite direction: W_new = W_old − α∇L, where α is learning rate. Small steps repeatedly move toward lower loss.

Underfitting: model is too simple, fails to learn the training data well (high train and test loss). Overfitting: model memorizes noise, fits training data perfectly but fails on new data (low train loss, high test loss). Regularization and validation help balance.

Deep learning uses neural networks with many hidden layers (depth). Depth is powerful because it allows hierarchical feature extraction—each layer builds on previous ones—enabling learning of very complex patterns that shallow networks cannot capture efficiently.