Glossary

Agent

A system where the model decides its own next steps in a loop, using tools, until a goal is met — as opposed to a workflow, where your code decides the steps. Agents are appropriate when the task is multi-step, hard to specify in advance, valuable, and recoverable from error. If any of those is false, use a workflow.

Agents SDK

OpenAI's open-source framework for multi-agent systems, providing agents, handoffs, guardrails, tracing, and a runner. Distinct from the Claude Agent SDK, which is Claude Code packaged as a library.

Adaptive thinking

Claude decides for itself how much internal reasoning a request needs, rather than being given a fixed token budget. Set with thinking: {"type": "adaptive"}. It replaced the older budget_tokens approach, which now returns a 400 on current models.

Batch API

Submit many requests for asynchronous processing at roughly half price. Results arrive in arbitrary order, so key them by your own ID rather than by position. The easiest large saving available on either platform.

Compaction

Server-side summarisation of earlier conversation context as it approaches the context limit. The API returns a compaction block you must pass back on subsequent requests — appending only the response text silently loses the state.

Context editing

Clearing stale tool results or thinking blocks from a transcript. Distinct from compaction: editing prunes, compaction summarises.

Context window

The maximum tokens in a single request — system prompt, tools, full conversation history, and the response being generated, all together. Current flagship models offer around 1M tokens. It is working memory for one request, not persistent memory.

Effort

A parameter controlling how much the model thinks and how much work it does per turn. On Claude: low through max, inside output_config, defaulting to high. On OpenAI, an equivalent reasoning-effort setting. Often a cheaper fix for shallow output than upgrading the model.

Embedding

A numeric vector representing the meaning of a piece of text, so that similar text produces nearby vectors. The basis of semantic search and RAG.

Eval / eval set

A collection of real inputs with known-good outputs or gradeable rubrics, used to measure whether a change helped. The single practice that most separates teams who ship AI reliably from teams who ship it hopefully.

Function calling / tool use

Describing functions to the model so it can request them with arguments. Your code executes them and returns results — the model never runs your code. Tool arguments are model output and must be treated as untrusted input.

Grounding

Supplying real source material — retrieved documents, search results, tool output — so the model answers from evidence rather than training recall. The structural fix for fabrication.

Guardrail

A validation step on input, output, or action that can block or redirect the model. Because it is code you control, it prevents an outcome rather than merely requesting good behaviour, which is why it works where a prompt instruction does not.

Hallucination

Confident, fluent, false output. A structural consequence of next-token prediction, not a bug to be instructed away. Reduce it with grounding, citations, and letting the model say it does not know.

Handoff

One agent transferring a conversation to another, more specialised agent. Several narrow agents connected by handoffs are more reliable and easier to debug than one agent with every tool.

Managed Agents

Anthropic's surface where the agent loop runs on Anthropic's orchestration layer and tools execute in a hosted per-session container. You create a versioned agent configuration once and reference it by ID from every session.

MCP (Model Context Protocol)

An open standard for exposing tools and data to a model. Write an MCP server once and any MCP-compatible client can use it — which is why MCP integrations survive model migrations, framework changes, and vendor changes.

Multimodal

Able to accept more than text — images, audio, documents. Current models support high-resolution vision, with image coordinates mapping directly to pixels.

Prompt caching

Reusing the processed form of a repeated prompt prefix, billed at roughly 10% of standard input price. It is a byte-exact prefix match, so a timestamp or user ID early in the prompt silently disables it. Verify with the cached-token count in the usage object.

Prompt injection

Instructions hidden in content the model reads — a web page, an email, a ticket, a document, a PR comment. The defence is not telling the model to ignore them; it is constraining what tools can do so that obeying an injection has an acceptable worst case.

Prefill

An older technique of ending the message array with a partial assistant turn to force an output shape. It returns a 400 on current Claude models. Structured outputs replace it, and the stop sequences and retry loops built around it become dead code.

RAG (retrieval-augmented generation)

Retrieving relevant documents and putting them in the prompt so the model answers from your data. Most RAG failures are retrieval failures, so measure retrieval recall separately before tuning any prompt.

Reasoning tokens

Tokens the model generates while thinking before answering. They bill at the output rate and are invisible in the response text, which makes them the usual explanation for a bill much larger than visible output suggests.

Re-ranking

Retrieving a larger candidate set cheaply, then using a model to rank the best few. One of the most reliable ways to improve retrieval quality without re-architecting.

Responses API

OpenAI's current default interface, combining Chat Completions with built-in tool use — web search, file search, computer use — in a single call. Chat Completions remains the portability choice.

Server-side tools

Tools that execute on the provider's infrastructure rather than yours — web search, web fetch, code execution. You declare them and the provider runs them. Their errors typically arrive as a successful response containing an error object, not as an exception.

Skills

Packaged task-specific instructions and files the model loads on demand, such as the pre-built document skills for spreadsheets, slide decks, and PDFs. Their value is progressive disclosure: the description sits in context, the full content loads only when relevant.

stop_reason

Why generation ended: finished naturally, hit the token cap, wants a tool result, paused, or refused. Always branch on it before trusting the response body — on a refusal the content may be empty.

Streaming

Receiving the response token by token rather than waiting for completion. Essential above roughly 16,000 output tokens to avoid HTTP timeouts, and the main determinant of how fast a product feels.

Structured outputs

Constraining the response to a JSON schema so it is guaranteed parseable. Guarantees the shape, never the truth — a schema requiring an integer returns one even when the source document was blank. Model uncertainty explicitly with nullable fields and a confidence score.

System prompt

Persistent instructions defining role, constraints, and output conventions. Also the front of the cacheable prefix, which is why interpolating anything dynamic into it is expensive.

Temperature

A sampling parameter controlling randomness on older models. Removed on current Claude flagship models, where sending it returns a 400. Steer behaviour with prompting and effort instead.

Token

The unit models read and write — roughly 3–4 characters of English, far fewer for code and non-English text. The unit of both cost and context. Count with the provider's own endpoint; OpenAI's tiktoken gives wrong numbers for Claude.

Tool runner

An SDK helper that drives the tool-calling loop for you. It yields each assistant message before tools execute, so approval gates, logging, and interception do not require hand-writing the loop.

Tool search

Letting the model discover relevant tools from a large library instead of loading every schema on every request. It appends schemas rather than swapping them, which preserves the prompt cache.

Vault

In Managed Agents, a store for credentials that Anthropic injects into outbound requests after they leave the sandbox. Code running in the container — including code the agent writes — cannot read them.

Vector store

A database of embeddings supporting nearest-neighbour search. If it holds documents with different access levels, permissions must be enforced at retrieval time, before anything reaches the model.