Neural Networks flashcards, generated for you.

A neural network learns to map inputs to outputs by adjusting weights in interconnected layers of neurons, mimicking how the brain processes information through connected units.

A weighted sum of inputs plus a bias, passed through an activation function: output = activation(w₁x₁ + w₂x₂ + ... + b). This allows the neuron to learn non-linear patterns.

Activation functions introduce non-linearity, allowing networks to learn complex patterns. Without them, stacking layers would just create a linear transformation (no added learning power).

The process of passing input data through each layer of the network, applying weights, biases, and activations, to produce a final prediction or output.

The loss function measures how far the network's prediction is from the true answer (e.g., mean squared error or cross-entropy). Minimizing it trains the network to make better predictions.

After computing the loss, we reverse through the network calculating how much each weight contributed to the error, then adjust weights in the direction that reduces loss (using the chain rule).

A gradient is the slope of the loss function with respect to each weight. Gradient descent moves weights in the opposite direction of the gradient (downhill) to minimize loss at learning rate η: w_new = w_old - η·∇L(w).

An iteration updates weights once using one batch of data. One epoch completes when all training data is processed. Batch size is the number of samples in one batch. If 1000 samples and batch size 100: 10 iterations per epoch.

Overfitting occurs when the network memorizes training data rather than learning general patterns, performing poorly on new data. Regularization penalizes large weights (L1/L2), and dropout randomly deactivates neurons during training to reduce co-adaptation.

Deep neural networks (DNNs) have many fully-connected layers for general tasks. Convolutional Neural Networks (CNNs) use weight-sharing filters for spatial data (images). Recurrent Neural Networks (RNNs) maintain hidden state across time steps for sequential data (text, time series).