Context Engineering: The Skill That Replaced Prompting

Short answer: in 2026, the highest-leverage skill in AI engineering is no longer writing a clever prompt — it’s context engineering: deciding exactly what information lands in the model’s context window at each step, and what you deliberately keep out. Prompt engineering asked “how do I phrase this?” Context engineering asks “what does the model need in front of it right now?” I build production RAG systems for a living, and this shift is the difference between a demo and something that ships.

What context engineering actually means

The term went mainstream in June 2025, when Andrej Karpathy popularized it (Shopify’s Tobi Lütke had used it publicly about a week earlier). Anthropic later put the cleanest definition on it: prompt engineering is about writing and organizing instructions, while context engineering is “the set of strategies for curating and maintaining the optimal set of tokens during inference — including all the information that lands there outside the prompt.” In plain terms: your system prompt, retrieved documents, tool outputs, memory and conversation history are all context, and managing that mix is the job.

Why prompt engineering is fading in 2026

Four things changed underneath us:

• Agents run in long loops, not single turns. A one-shot prompt is a rounding error next to an agent that accumulates tool outputs and intermediate reasoning across dozens of steps. Context is now a living thing you manage, not a sentence you polish once.
• Context windows exploded — but bigger isn’t better. Frontier models now ship 1,000,000-token windows (Claude, Gemini, GPT-5.5). Yet Chroma’s “Context Rot” study tested 18 models and found accuracy degrades non-uniformly as input grows — often well before the stated limit. Anthropic now bakes the term straight into its docs: “as token count grows, accuracy and recall degrade.” The job became managing scarce attention, not phrasing.
• Retrieval and memory became the real levers. The winning move isn’t a cleverer sentence — it’s pulling the right chunk just-in-time, summarizing history before it overflows, and persisting memory across sessions.
• Tool selection is part of the prompt surface now. When an agent wields many tools, which schemas you load and when drives behavior as much as the instruction text.

The context-engineering toolkit

Here’s the working set, and where my own stack plugs in. ★ marks the parts I run in production.

• Retrieval / RAG ★ — fetch only the relevant chunks at query time instead of stuffing everything in. I run this on pgvector inside Postgres, so vectors and business rows live in one database with one backup story.
• Embeddings & chunking ★ — retrieval quality is capped by embedding quality. I embed with Voyage; the embedding model moves RAG quality more than the LLM does.
• Re-ranking — a sharper second pass reorders the top results so the best evidence sits closest to the question. This is your main defense against context rot.
• Context-window budgeting — treat the window as a finite attention budget and spend it on the smallest set of high-signal tokens.
• Compaction & summarization — summarize and re-initialize long histories before they hit the limit.
• Memory — persist state outside the window and reload it, for cross-session continuity without paying for full history on every call.
• Tool / function selection ★ — load only the schemas a step needs. In LangGraph I gate which tools each node can even see.
• Context isolation / sub-agents — hand a focused sub-task to a specialized agent that returns a condensed summary, keeping the parent’s context clean.

What I stopped doing in 2026

Most “prompt engineering” tricks people still teach are now dead weight inside the context window:

• “You are an expert…” role-play preambles. Identity priming barely moves accuracy on frontier models and spends tokens in a budget you’re trying to shrink.
• Incantations and bribes (“I’ll tip you”, “or you’ll be penalized”). Folklore with no durable, measured benefit; modern models follow plain instructions more literally, so the theatrics just add noise.
• Megaprompts — stuffing it all in “just in case.” More tokens is not more accuracy. Context rot means recall drops as you pile on. Curate, don’t dump.
• Hard-coded few-shot for everything. Static examples pay their token cost on every call; retrieval pulls the relevant example just-in-time. Few-shot still helps for format — not as a dumping ground.
• Jailbreak-style hacks. Unreliable against 2026 safety training and irrelevant to production reliability.
• Tuning by vibes. Hand-tweaking wording with no test set is guesswork.

The part nobody gets to skip: evals

If context engineering has one non-negotiable, it’s this: measure, don’t guess. I gate changes on Ragas — faithfulness and context-precision on a frozen test set — not on gut feel. A retrieval tweak that “feels” better but quietly drops precision is a regression you shipped blind. The whole discipline only works because you can prove a change helped.

FAQ

Is prompt engineering dead?

No — it’s absorbed. Clear instructions still matter, but they’re one input among many. In 2026 the leverage moved to what you put in the window, not just how you word it.

Who coined the term “context engineering”?

Andrej Karpathy popularized it in June 2025; Shopify’s Tobi Lütke used it publicly about a week earlier. Within roughly a month it had a 1,400-paper academic survey behind it.

Is context engineering just RAG?

RAG is a big piece, not the whole thing. Context engineering also covers memory, compaction, tool selection, context isolation and window budgeting. RAG is how you get the right documents in; the rest is how you keep the window clean.

Do bigger context windows make this obsolete?

The opposite. 1M-token windows make discipline more important, because accuracy rots long before the limit. A big window is rope; context engineering is knowing how much of it to use.

Building an AI agent or RAG system and want it to actually hold up in production? Let’s talk.