Flashcards · Large Language Models · Free
Large Language Models flashcards, generated for you.
Example Large Language Models study cards to learn from right now — then generate a full set from your own notes (plus a practice quiz) and export to Quizlet or Anki. Free, no account needed.
Example Large Language Models flashcards
What is the core idea behind how large language models work?
LLMs predict the next word in a sequence by learning patterns from massive amounts of text. They work by computing probabilities: given previous words, what word comes next?
Why is 'transformer' a better name than 'recurrent' for modern LLMs?
Transformers process all words in parallel (not sequentially), allowing them to learn long-range relationships efficiently. They 'transform' input sequences into meaningful representations simultaneously rather than step-by-step.
What does 'attention' mean intuitively in the attention mechanism?
Attention lets the model focus on relevant parts of the input when processing each word. For example, when predicting the next word after 'The bank manager...', attention can highlight 'bank' and 'manager' as most relevant.
Why do LLMs need positional encoding?
Without it, the model can't distinguish word order (since attention processes all words in parallel). Positional encodings add information about each word's location so 'dog bites man' differs from 'man bites dog'.
What is the mathematical purpose of softmax in attention?
Softmax converts raw attention scores into a probability distribution (all values sum to 1). It amplifies large scores and suppresses small ones, so the model heavily 'attends to' the most relevant words.
What does 'token embedding' do?
It converts discrete words/subwords into high-dimensional vectors (e.g., 768 dimensions) that capture semantic meaning. Similar words end up close together in this space; similar vectors can be operated on mathematically.
Why do transformers use multiple attention heads instead of one?
Different heads learn different types of relationships. One head might focus on syntax, another on semantics, another on long-range dependencies. Multiple heads learn complementary patterns in parallel.
What is the mathematical formula for scaled dot-product attention?
Attention(Q, K, V) = softmax(QK^T / √d_k) V, where Q=queries, K=keys, V=values, and d_k is key dimension. Dividing by √d_k prevents scores from becoming too extreme before softmax.
How does a feedforward network in a transformer layer enhance learned patterns?
After attention, a two-layer feedforward network (with a nonlinearity like ReLU) applies learned transformations independently to each position. It provides capacity for non-linear feature combinations without attending to other tokens.
Why is layer normalization applied before (not after) attention/feedforward layers in modern transformers?
Pre-normalization stabilizes training by normalizing inputs to each component, preventing extreme activation values. This allows deeper models to train more stably than post-normalization, improving convergence and final performance.
Make your own Large Language Models study set
Flashcards for related topics
Studying Large Language Models to build with AI? MindloomHQ turns it into real skills — structured courses, agent projects, and certificates.
Explore MindloomHQ →