Deep Learning flashcards, generated for you.

Deep learning learns hierarchical representations by stacking layers that progressively extract higher-level features from raw input (e.g., pixels → edges → shapes → objects).

Multiple layers allow efficient feature composition: each layer learns simple patterns, which are combined by later layers into complex patterns. This is more parameter-efficient and generalizable than learning everything at once.

A neuron computes a weighted sum of inputs plus a bias, then passes it through an activation function. Intuitively: it's a learnable feature detector that fires (activates) when its favorite input pattern appears.

An activation function (e.g., ReLU, sigmoid) introduces non-linearity. Without it, stacking layers would just compute linear transformations, limiting the network to learning only linear relationships.

Backpropagation computes how much each weight contributed to the final error, working backward through layers using the chain rule. This tells us how to adjust weights to reduce error.

A loss function measures the difference between predicted and true outputs (e.g., mean squared error, cross-entropy). Training minimizes this loss by adjusting weights to reduce prediction error on training data.

Parameters (weights and biases) are learned during training via backpropagation. Hyperparameters (learning rate, layer size, activation function) are set before training and control how learning happens.

In very deep networks with certain activation functions (e.g., sigmoid), gradients shrink exponentially as they backpropagate, making early layers learn very slowly. ReLU and batch normalization help mitigate this.

Overfitting occurs when a model memorizes training data rather than learning generalizable patterns, failing on new data. Regularization (L1/L2 penalties, dropout, early stopping) penalizes complexity to improve generalization.

CNNs exploit spatial locality via convolutional filters (good for images). RNNs process sequences with hidden state memory (good for time series). Transformers use attention to weigh all input positions (good for sequences and many domains).