Neural Networks flashcards, generated for you.

A neural network learns to map inputs to outputs by adjusting internal parameters (weights). It mimics how biological neurons connect: each node receives signals, processes them, and passes results forward.

A neuron computes a weighted sum of its inputs, adds a bias term, and applies a nonlinear function (activation function) to produce an output. Formula: output = activation(Σ(weight × input) + bias).

Activation functions introduce nonlinearity. Without them, stacking layers would just create linear transformations, making the network unable to learn complex patterns. Common ones: ReLU, sigmoid, tanh.

A layer is a group of neurons that process information together. Input layer receives raw data, hidden layers learn features, output layer produces predictions. Neurons in one layer connect to all neurons in the next layer (in a fully connected/dense network).

Backpropagation calculates how much each weight contributed to the error. Starting from the output error, it works backward through the network (using the chain rule) to assign 'blame' to each weight, enabling gradient-based updates.

A loss function quantifies how wrong the network's predictions are compared to true labels. Common examples: Mean Squared Error (MSE) for regression, Cross-Entropy for classification. Lower loss means better performance.

Gradient descent iteratively adjusts weights in the direction that reduces loss. It computes the gradient (slope) of loss with respect to each weight and takes a small step opposite to that gradient, controlled by learning rate.

Overfitting occurs when a network memorizes training data too well but fails on new data. It learns noise and specific details instead of generalizable patterns. Indicated by low training loss but high validation loss.

Regularization adds a penalty for large weights (L1/L2) or randomly drops neurons during training (dropout), forcing the network to learn simpler, more robust features rather than fitting noise.

Iteration: one weight update (forward pass + backprop on one batch). Batch size: number of samples per iteration. Epoch: one complete pass through entire training data (multiple iterations). Example: 1000 samples, batch size 100 = 10 iterations per epoch.