Deep Learning flashcards, generated for you.

Deep learning learns hierarchical representations by stacking layers that progressively extract higher-level features from raw input (e.g., edges → shapes → objects).

Multiple layers enable compositional learning: early layers learn simple patterns, middle layers combine them into more abstract features, and deeper layers solve complex tasks efficiently with fewer parameters than a single layer would need.

A neuron is a simple computational unit that takes weighted inputs, sums them, adds a bias, and applies a non-linear activation function to produce an output that signals to the next layer.

Activation functions introduce non-linearity. Without them, stacking layers is mathematically equivalent to a single linear transformation, so the network cannot learn complex patterns.

Backpropagation computes gradients of the loss with respect to every weight by applying the chain rule backwards through the network, enabling efficient gradient descent optimization.

During backprop through many layers with sigmoid/tanh activations, gradients multiply together and become exponentially smaller, causing early layers to learn very slowly or not at all.

Skip connections allow gradients to flow directly across layers without multiplication, preventing vanishing gradients and letting the network learn identity mappings when needed, enabling training of much deeper networks.

Batch normalization normalizes layer inputs to zero mean and unit variance within each batch. This stabilizes learning (reduces internal covariate shift), allows higher learning rates, and reduces dependence on weight initialization.

Underfitting: model is too simple and misses patterns in both training and test data (high bias). Overfitting: model memorizes training data and fails on test data (high variance). Deep networks are prone to overfitting without regularization.

Dropout randomly disables neurons during training, forcing the network to learn redundant representations. L2 regularization penalizes large weights, pushing the network toward simpler solutions. Both reduce effective model complexity and generalization error.