Claude Opus 4 + RAG + Knowledge Graph: A Real-World Architecture for AI Novel Writing

In an era overflowing with AI-assisted creative tools, getting large language models to produce coherent, consistent stories that hold up across long-form narratives remains a largely unsolved problem. In a recent episode of MVP Unplugged, Michael — a Microsoft MVP with nearly 20 years of experience and a seasoned open-source contributor — shared his open-source project AI Story Builders, three years in the making. He demonstrated how to combine Claude Opus 4, Embeddings, and a Knowledge Graph to build a genuinely usable AI fiction-writing tool.

This article isn't just a product overview — it's an architectural methodology for how to design context structures for LLMs.

A Project Born from Failure

Michael is candid: AI Story Builders is "a story about failure." When ChatGPT first launched three years ago, he wanted to build something he actually needed — a tool to help write novels. The initial idea was simple: store characters, locations, and other elements in a database, then let AI help continue the story.

The harsh reality was that models at the time "couldn't write a single decent page, let alone a full chapter." He spent six months churning through one proof of concept after another — all failures. The most straightforward approach — dumping the entire story into a prompt and asking the AI to "write the next chapter" — simply didn't work. The model would either produce incoherent output, or text that read smoothly but was completely hollow.

His core conclusion is remarkably insightful:

AI is just an algorithm, a computation — it doesn't actually think. What you put in is what you get out.

The real question, therefore, was never "how do you get AI to write" — it was always "how much precise context can you feed it." That insight became the foundational design principle for the entire project.

Story Structure Design: Why the Timeline Comes First

Michael's most valuable insight is his structured breakdown of what a "story" actually is as an object. Most people assume the core elements are characters and chapters — but in practice, he discovered a counterintuitive order of priority.

The Timeline Is the Primary Element

He argues that a story doesn't start with characters — it starts with the Timeline. Even in stories with flashbacks or non-linear structures, time remains the underlying control logic. If a character is five years old in a flashback, that "temporal anchor point" determines every attribute they have at that moment. Everything else must attach to the timeline.

Locations and Characters Follow

Second comes Location. Whether the world is real or fictional, "when" and "where" together form the skeleton of the story. Even if a location is mysterious to the reader, it still exists as a distinct, identifiable entity in the data structure.

Only after the timeline and locations are established do Characters come into play — along with their Attributes such as hair color, backstory, and so on. Character attributes are also bound to the timeline (e.g., "red hair" might only apply during a specific period).

This Timeline → Location → Character → Attribute hierarchy essentially transforms an unstructured creative task into relational data that can be managed programmatically and retrieved with precision — laying the groundwork for RAG retrieval and knowledge graph construction later.

Technical Implementation: Text Files + Embeddings + RAG

Interestingly, AI Story Builders doesn't rely on a dedicated vector database. Instead, it stores everything in plain text files.

In the user's documents directory, each story gets its own folder, subdivided into chapter subfolders, with each paragraph stored as a separate text file. Opening any paragraph file reveals three parts:

• The characters involved in that paragraph
• The prose body of the paragraph
• The embeddings (vector representation) for that paragraph

Embeddings are the technology that converts text into high-dimensional numerical vectors, with the core idea being that semantically similar text sits closer together in vector space. Embedding models from OpenAI, Cohere, and others can compress a passage of text into a floating-point array of hundreds to thousands of dimensions. AI Story Builders serializes these vectors directly into plain text files, eliminating the need for dedicated vector databases like Pinecone or Weaviate. For small-to-medium story libraries, loading vectors into memory at runtime for batch comparison works perfectly well.

At runtime, the application reads these embedding vectors and computes relevant paragraphs using cosine similarity — a measure of the angle between two vectors, ranging from -1 to 1, where values closer to 1 indicate greater semantic similarity. The results are then used to dynamically assemble the prompt sent to the AI. This is the standard RAG (Retrieval-Augmented Generation) pipeline.

RAG's core logic is to externalize "memory" from model parameters into a dynamically retrievable database: ① slice and vectorize external knowledge; ② retrieve the most relevant segments when a user query comes in; ③ inject the retrieved results into the prompt before passing it to the model. This allows the model to handle story libraries far larger than its context window without loading the full text each time.

During actual writing sessions, the tool enforces one ironclad rule: have the AI write only one chapter at a time. When the user clicks "Continue" or describes a request via the AI button, the system automatically feeds the model the previous chapter's content, the characters and locations involved in the current chapter, the bound timeline, and all other relevant context. Michael emphasizes that even a frontier model like Claude Opus 4 tends to "go off the rails" when asked to write too much at once.

He also demonstrated a practical feature: after importing The Great Gatsby, the tool automatically extracts characters and locations, and can even "continue" the novel from the final paragraph. The demo also revealed some rough edges — "Daisy" and "Daisy Buchanan" were incorrectly treated as two separate characters. This kind of Entity Disambiguation problem — determining whether different string representations refer to the same real-world entity — is still being actively refined.

From RAG to Knowledge Graph: Letting the AI Actively Explore Information

The most recent and most exciting development in the project is the introduction of a Knowledge Graph. Michael believes it made the whole program "ten times better."

A knowledge graph is a graph structure built on the fundamental unit of "entity–relation–entity" triples, consisting of nodes and edges: nodes represent real-world entities (characters, locations, events), while edges describe named relationships between them. Compared to relational databases, graphs are naturally suited for multi-hop association queries — for example, "find all characters who interacted with Watson and appeared on Baker Street" requires only edge traversal in a graph, whereas SQL would demand multiple JOINs. This is precisely why knowledge graphs outperform plain RAG in narrative consistency checking.

Michael first built an application that scans existing story files and generates raw JSON data for the knowledge graph, then visualizes it. Using the Sherlock Holmes stories as an example, the graph's nodes represent characters, and the edges carry relationship information such as "this character interacted with that character," resolved down to the paragraph level.

The Key Breakthrough: Exposing the Knowledge Graph as a "Tool" to the Model

The real architectural upgrade is this: the graph isn't simply stuffed into the prompt. Instead, it is exposed to frontier models as a set of tools.

Tool Use / Function Calling is a core capability extension mechanism for modern LLMs: developers declare a set of callable external functions and their parameter schemas to the model, which can then autonomously decide during inference when to call which function, with what parameters, and continue generating based on the return values. This evolves the model from "passively answering" to "actively exploring" — the model can issue multiple graph queries in sequence: first checking character relationships, then checking for timeline conflicts, then verifying location ownership, forming a multi-step reasoning chain similar to the ReAct (Reasoning + Acting) framework.

Michael explains the fundamental difference from traditional RAG:

With RAG, there's a hard ceiling on what you can fit into a prompt. With graph tool calls, the model can query as many times as it needs — so it can uncover everything it truly requires.

A newly added chat window makes this capability directly accessible. You can ask "list all interactions between Watson and Holmes in Chapter 1," and the model returns a detailed answer that plain RAG couldn't retrieve in a single pass.

He also opened up a "write-back" tool: when the AI detects a contradiction in the story, it can directly update the chapter file to fix the issue. This closes the loop on the entire workflow — use the knowledge graph to verify consistency, then feed the verified results back as context to write more accurate new content.

Universal Lessons for AI Application Developers

As a fully open-source project under the MIT license (available on both GitHub and the Microsoft Store for free), AI Story Builders offers value far beyond fiction writing.

Michael presents a transferable methodology any AI application can adopt:

1. Identify the entities and relationships (edges) in your domain and construct a knowledge graph;
2. Expose the graph to the LLM as tools rather than stuffing it wholesale into the context;
3. Let the model perform multi-turn tool calls to achieve deeper information retrieval and manipulation than plain RAG allows.

This maps directly onto the current technical trend toward Agents and tool use: when a model is capable enough, rather than agonizing over how to compress context, give it a set of tools it can actively explore. At its core, this upgrades the LLM from a "reading comprehension machine" into a "reasoning engine" that can autonomously navigate a structured knowledge base.

Conclusion

The three-year evolution of AI Story Builders is almost a microcosm of AI application development itself: from piling massive prompts into weaker models, to harnessing the power of Claude Opus 4, to rebuilding context architecture with a knowledge graph.

The reminder it leaves for every AI developer remains constant — no matter how powerful the model, it cannot escape the iron law that input determines output. The real engineering moat isn't which model you call — it's how you organize, retrieve, and deliver precisely the right context on demand. From flat text retrieval (RAG) to structured graph traversal (Knowledge Graph + Tool Use), this evolutionary path is itself an architectural reference note written for every AI engineer.