New Book 'Context Engineering' Released: AI Systems Often Fail Because of Context

Introduction: AI Failures Are Often Not the Model's Fault

Most AI systems fail in production not because the wrong model was chosen, but because the context is wrong. This is the core argument of the new book Context Engineering: Designing AI That Understands, Remembers, Acts, and Learns. The authors point out that context engineering is not an entirely new concept—it simply applies the disciplines we have practiced for decades in configuration management, systems engineering, and domain-driven design to LLMs, a new class of system.

With the arrival of the LLM era, many engineering disciplines were set aside. Context was reduced to "prompts," something to be repeatedly tweaked rather than an engineering asset requiring version control, validation, and governance. This book attempts to correct that deviation.

Why AI Software Fails Differently

Traditional software failures typically manifest as crashes, exceptions, or clearly missing functionality. AI system failure modes are far more insidious—context drift is a typical example. The model does not throw an error, and the output looks reasonable, but as the context gradually diverges from expectations, system behavior degrades imperceptibly.

This failure mode imposes new requirements on engineering practice. You cannot rely solely on unit tests and monitoring alerts to catch problems; you also need to design observability for the context itself. The book breaks AI systems down into multiple components: prompts, knowledge, retrieval, policy, and the model's own emergent behavior. Each part needs to be treated independently while also being governed in coordination.

Context drift refers to the gradual accumulation of noise, outdated facts, or incorrectly inferred content within the context window of multi-turn interactions or long-running AI systems, causing model outputs to systematically deviate from the original intent. It differs from model hallucination—hallucination is a factual error within a single generation, whereas drift is a bias that accumulates gradually across sessions or long reasoning chains. Causes of drift include: retrieved documents becoming less timely, early errors in conversation history being repeatedly referenced, and tool call results not being updated in time. Addressing drift requires engineering measures such as context versioning, periodic re-retrieval, explicit state snapshots and diff comparison, and evidence-provenance-based regression testing.

Knowledge Graphs and Deterministic Retrieval: The Infrastructure of Context

The book devotes considerable space to discussing the value of knowledge graphs as context infrastructure, along with a key judgment: when deterministic search outperforms vector embeddings. Vector retrieval excels at fuzzy semantic matching, but in scenarios requiring precise recall, interpretability, and stability, deterministic retrieval based on knowledge graphs is often more reliable.

This view is directly related to Terraphim, a project the author is involved in. Terraphim provides role-aware search, knowledge graphs, and deterministic retrieval capabilities, and the author explicitly positions it as "context infrastructure" rather than an ordinary feature. This positioning reflects the book's consistent stance: context is not an appendage to prompts but a system layer requiring dedicated architectural design.

Deterministic retrieval refers to a retrieval method that returns exactly the same result set every time given the same query and the same knowledge base. Typical implementations include precise path queries based on graph traversal, exact matching based on primary keys/aliases, and structured filtering under rule constraints. In contrast, vector embedding retrieval relies on approximate nearest neighbor search, and results are affected by embedding model versions, index parameters, and floating-point errors, potentially producing differences across runs. The advantages of deterministic retrieval lie in reproducibility, interpretability, and auditability: every recalled result can be traced to specific graph nodes and relationship paths, making it easy to prove in compliance scenarios "why this piece of evidence was selected." It is especially suited to tasks requiring precision, such as entity resolution, permission filtering, and fact verification.

Making AI Decisions Auditable: Judge Harnesses and Evidence Lineage

A core pain point of enterprise-grade AI applications is that AI decisions are difficult to audit and trace. The book proposes mechanisms such as "judge harnesses," typed decision models, and evidence lineage, with the goal of making every AI decision inspectable—not only knowing what the result is, but also tracing which evidence it was based on and what logical path it went through.

This part is especially important for enterprises that need to meet compliance requirements. The book also covers topics such as validation gates, multi-agent context management, and enterprise governance, forming a complete framework from technology to organization.

A judge harness is an evaluation and review framework built around model decisions, typically including: input context snapshots, structured records of model outputs, a set of configurable decision rules or review models, and baseline datasets for regression comparison. Its role is not to replace human review but to turn "whether a decision is reasonable" into repeatable checks, such as whether the evidence is sufficient, whether the reasoning path is consistent with policy, and whether the output conforms to type constraints. Evidence lineage records the original data sources, retrieval paths, and processing steps that each decision depends on, forming a complete chain from conclusion back to evidence. Together, the two enable AI decisions to be reproduced and cross-examined item by item in compliance audits.

A Practical Guide for Technical Leaders

The book has 25 chapters in total. In addition to the technical content above, it also covers practical topics such as token budgets, context compression, and skill architecture. The final chapter is dedicated to how to establish context engineering practices within a team.

The book's target audience is senior developers, engineering managers, and architects—those who need to make AI run reliably in production, not just impress in demos. From configuration management to systems engineering to DDD, software engineering has accumulated a rich tradition of discipline; the book's argument is that it is time to bring these disciplines back into AI system development.

For teams pushing AI systems into production, the book's framework provides a checklist worth comparing against: Is your context versioned? Have your retrieval strategies been validated? Are AI decisions traceable? The answers to these questions often determine whether an AI project can truly land.

Token budget refers to the maximum token quota allocated to different context components (system prompts, retrieved documents, conversation history, tool outputs, etc.) within a single request or a single session. It is essentially budget management for context resources: the total is limited by the model's context window, and the quota for each part determines the information trade-off strategy. A reasonable token budget needs to be dynamically adjusted according to task type—for example, factual Q&A should prioritize the quota for retrieved evidence, while multi-turn task-oriented dialogue needs to reserve space for history summaries. Supporting measures include context compression (summarizing history or extracting key entities) and tiered caching (solidifying stable, unchanging parts into reusable prefixes) to maintain information density and consistency within a limited window.

One-line Summary

AI production failures mostly stem from context rather than the model; context infrastructure must be rebuilt with engineering discipline.

Paragraph Summary

The article introduces the core argument of the book Context Engineering: the main cause of AI system failures in production is context errors, not model selection errors. The author argues that context engineering is not a new concept but the application of existing disciplines such as configuration management, systems engineering, and domain-driven design to LLM systems. The book analyzes context drift, an insidious failure mode, and advocates decomposing and governing prompts, knowledge, retrieval, policy, and model behavior separately; argues that knowledge graphs and deterministic retrieval outperform vector embeddings in scenarios requiring precise recall and interpretability; and proposes mechanisms such as judge harnesses, typed decision models, and evidence lineage to make AI decisions auditable and traceable. Across 25 chapters, the book also covers token budgets, context compression, multi-agent context management, and enterprise governance, offering senior developers, engineering managers, and architects a practical checklist from technology to organization.