Prompt Engineering flashcards, generated for you.

Prompt engineering is about crafting input text to guide an AI model toward producing better outputs. Just like asking a colleague a clear question gets better answers than a vague one, specific, well-structured prompts elicit more useful model responses.

Language models predict the next token based on patterns in training data. The exact wording, context, and structure of a prompt activate different learned patterns, steering the model toward relevant knowledge and reasoning paths rather than unhelpful ones.

In-context learning means the model learns from examples and instructions provided in the prompt itself (not from retraining). It works because the prompt positions relevant patterns at the front of the model's attention, making those patterns more likely to influence the next predictions.

Chain-of-thought prompting asks the model to show step-by-step reasoning before answering. This works because generating intermediate reasoning tokens creates a 'path' through the model's knowledge that makes complex reasoning more likely, rather than jumping directly to an answer.

Temperature controls randomness in token selection: low temperature (e.g., 0.1) makes the model pick highest-probability tokens (deterministic, focused), while high temperature (e.g., 1.0) adds randomness. Lower temps suit precision tasks; higher temps suit creative tasks. It's a scaling parameter on the softmax probability distribution.

Tokens are the smallest units a language model processes—roughly words or subword chunks (e.g., 'running' = ['run', 'ning']). Prompt engineering must account for token limits and token cost, since models have max context windows (e.g., 4K, 128K tokens) and token counts directly affect pricing.

The system prompt is initial instructions that set the model's behavior, tone, and role (e.g., 'You are a helpful tutor'). The user prompt is the actual request. System prompts occupy early context and have outsized influence on behavior; they establish the 'frame' before any user input.

Few-shot prompting provides k exemplars in the prompt before the query: (example₁_input, example₁_output), ..., (exampleₖ_input, exampleₖ_output), then (query_input). This shifts the model's conditional probability distribution P(output|prompt) by conditioning on similar patterns, boosting accuracy on the target task via in-context learning.

Prompt sensitivity is the degree to which small changes in wording cause large changes in output. It reflects that neural networks can have sharp decision boundaries. Robust prompts produce consistent outputs across minor phrasings; adversarial robustness in prompting means outputs remain stable despite small perturbations or paraphrases.

Prompt injection is an attack where untrusted user input is embedded in a prompt, causing the model to follow malicious instructions hidden in that input, overriding intended system constraints. Mathematically, it exploits the fact that the model has no hard boundary between 'system context' and 'user data'—all tokens contribute equally to P(output|full_prompt).