Why this matters

For years, "prompt engineering" was the headline skill for working with LLMs. Write a clearer instruction, add a few examples, and watch the output improve. That framing made sense when the entire interaction was a single user message plus a system prompt.

Modern LLM applications are different. A production system assembles context from five or six distinct sources before the model ever sees the request. The system prompt carries behavioral instructions. Conversation history carries session memory. A retrieval step fetches relevant documents. Tool results from previous turns provide live data. A user profile injects personalization. Each of these contributes tokens, and the model's response quality depends on how well all of those pieces fit together.

Context engineering is the discipline of designing, assembling, and managing the entire information environment an LLM operates in. Prompt engineering is one input to that discipline. RAG, memory management, and tool result formatting are the others.

Core concepts

The context stack

Think of what the model receives as a stack of layers, assembled before every call:

1. System prompt — static behavioral instructions, persona, constraints, output format requirements
2. User message — the current turn's input
3. Conversation history — prior turns, summarized or verbatim
4. Retrieved documents — chunks from a vector store or search result
5. Tool results — outputs from previous function calls in the current agent loop
6. Dynamic injections — user profile, current date/time, feature flags, environment context

Each layer competes for tokens. Decisions made at each layer affect the quality of the model's response at every other layer.

Why "write a better prompt" is no longer enough

The failure modes of modern LLM systems are rarely about instruction clarity. They are about context composition:

• The relevant document was retrieved but the history already consumed so many tokens that it was truncated
• The tool result was injected verbatim but it was 8,000 tokens of raw JSON the model could not efficiently attend to
• The system prompt was excellent but a conversation that ran 30 turns diluted its effect with irrelevant history
• The retrieval was semantically correct but the formatting made it look like noise to the model

These are context engineering failures, not prompt engineering failures.

Connecting the disciplines

Context engineering is the unifying frame for prompting, RAG, memory management, and tool use. Each of these is a sub-discipline. Context engineering is the meta-discipline that governs how they interact.

Code: ContextBuilder

from dataclasses import dataclass, field
from enum import IntEnum


class Priority(IntEnum):
    SYSTEM = 0       # highest priority, never truncated
    USER_INPUT = 1
    TOOL_RESULTS = 2
    RETRIEVAL = 3
    HISTORY = 4      # lowest priority, first to be dropped


@dataclass
class ContextLayer:
    role: str
    content: str
    priority: Priority
    source: str = ""


@dataclass
class ContextBuilder:
    max_tokens: int = 8000
    layers: list[ContextLayer] = field(default_factory=list)

    def add(self, role: str, content: str, priority: Priority, source: str = "") -> "ContextBuilder":
        self.layers.append(ContextLayer(role, content, priority, source))
        return self

    def _estimate_tokens(self, text: str) -> int:
        return max(1, len(text) // 4)

    def build(self) -> list[dict]:
        sorted_layers = sorted(self.layers, key=lambda l: l.priority)
        budget = self.max_tokens
        selected: list[ContextLayer] = []

        for layer in sorted_layers:
            cost = self._estimate_tokens(layer.content)
            if cost <= budget:
                selected.append(layer)
                budget -= cost
            elif layer.priority <= Priority.USER_INPUT:
                selected.append(layer)
                budget -= cost

        order = [Priority.SYSTEM, Priority.RETRIEVAL, Priority.HISTORY,
                 Priority.TOOL_RESULTS, Priority.USER_INPUT]
        selected.sort(key=lambda l: order.index(l.priority))
        return [{"role": l.role, "content": l.content} for l in selected]


# Usage
builder = ContextBuilder(max_tokens=6000)
builder.add("system", system_prompt, Priority.SYSTEM)
builder.add("user", user_profile_summary, Priority.RETRIEVAL)
for doc in retrieved_docs:
    builder.add("user", f"Reference: {doc}", Priority.RETRIEVAL)
for turn in conversation_history:
    builder.add(turn["role"], turn["content"], Priority.HISTORY)
builder.add("user", current_user_message, Priority.USER_INPUT)
messages = builder.build()

The key insight: each layer declares its priority when added. When the budget is tight, lower-priority layers are dropped automatically.

Common mistakes

Treating context as append-only. Most starter implementations just append to a messages list. When the list grows beyond the context window, the call fails.

All layers have equal priority. Without a priority system, you have no principled way to decide what to drop.

Not logging what was dropped. When context is truncated, the model behaves differently and you will not know why.

Ignoring formatting overhead. Markdown headers, bullet points, and code fences all consume tokens.

Try it yourself

1. Extend ContextBuilder with a build_with_report() method that returns both the message list and a summary of tokens consumed per source.
2. Add a CRITICAL priority level above SYSTEM for content that must always appear.
3. Take a real multi-turn conversation and map each message to one of the five context stack layers. Which layers were the largest?