The Rebirth of Xanadu: A Hypertext Dream Built for AI Agents

A Dream Spanning Half a Century

In the long history of the internet, one project has been called "the longest-running software development epic" — Xanadu. Proposed by computing pioneer Ted Nelson in 1960, it predated Tim Berners-Lee's invention of the World Wide Web by a full thirty years.

Ted Nelson and the Birth of Hypertext

Ted Nelson is an American philosopher, sociologist, and computing pioneer, born in New York in 1937. As early as the 1960s, he foresaw that computers would fundamentally transform how humans process information. In 1963, he introduced the concept of "hypertext" — a nonlinear, web-like way of organizing text that allows users to freely navigate between different text fragments via links. This concept broke the linear reading model of traditional printed books and laid the theoretical groundwork for how information would later be organized on the internet. Nelson's vision went far beyond simple link-based navigation; he envisioned a complete literary system (docuverse) in which all human knowledge would be interconnected in a structured, traceable way.

Nelson coined the terms "hypertext" and "hypermedia," and sketched out a vision for an information network far grander than today's Web. His ideas profoundly influenced the later development of the internet, even though his Xanadu project itself was never successfully commercialized.

Yet Xanadu never truly came to fruition. Many of its core ideas were so far ahead of the technology available at the time that it was derided as "vaporware" — software perpetually in development. Recently, a post titled Xanadu was waiting for agents sparked discussion on Hacker News, garnering 59 upvotes and 24 comments. The article put forward an intriguing argument: the reason Xanadu was "born ahead of its time" may be that its true users were never humans — they were AI agents.

The Fundamental Differences Between Xanadu and the World Wide Web

To understand this argument, we first need to clarify how Xanadu differs from the World Wide Web we know today.

The Technical Architecture of the World Wide Web

Tim Berners-Lee invented the World Wide Web in 1989 at CERN (the European Organization for Nuclear Research). The Web is built on three core technologies: HTML (HyperText Markup Language) for creating web content, HTTP (HyperText Transfer Protocol) for transmitting data between servers and clients, and URLs (Uniform Resource Locators) for uniquely identifying and locating network resources. The design philosophy behind this architecture is simplicity, openness, and decentralization. Anyone can create web pages and set up servers without approval from a centralized authority. It was precisely this minimalism and openness that allowed the Web to spread so rapidly.

However, this design also introduced shortcomings. Today's Web uses unidirectional links: you click a link to jump to another page, but the target page has no idea who linked to it, and broken links (404 errors) are a fact of life. These issues were tolerable in an era where humans were the primary users, but they pose serious obstacles for AI systems that require precision and verifiability.

Bidirectional Links and Content Permanence

Xanadu's design includes several revolutionary features:

Bidirectional links: Every link is traceable in both directions — source and target are each aware of the other's existence. This stands in stark contrast to the Web's unidirectional links. In Xanadu's system, when you create a link to another document, the target document automatically "knows" it has been referenced and can display all backlinks pointing to it. This creates a true knowledge network, rather than a one-way flow of information pointers.

Content never disappears (no broken links): Through an ingenious content-addressing system, Xanadu guarantees that referenced content is always accessible.

A Solution to Link Rot

Link rot — or link decay — is a chronic ailment of the World Wide Web. Research shows that roughly 11% of links on the web break every two years on average. This happens because Web links point to a specific location on a server, and site owners can delete, move, or rename files at any time, causing URLs to return 404 errors. For academic citations, legal documents, news reports, and other content that needs long-term preservation, this is a serious problem. The Internet Archive's Wayback Machine attempts to mitigate this by saving web page snapshots, but that is only a retroactive remedy.

Xanadu's design fundamentally solves this problem through content addressing rather than location addressing — a piece of content's identifier is based on the content itself, not its storage location, so content can move between servers without affecting accessibility. For AI agents, link rot causes fractures in knowledge graphs, undermining the accuracy and reliability of reasoning.

Transclusion: This is the concept Nelson cherished most.

Transclusion Explained

Transclusion is one of the core concepts Nelson proposed in the Xanadu project. It refers to referencing part of another document within your own document — not by simply copying and pasting, but by creating a dynamic, source-attributed reference. Like citing original text in an academic paper, transclusion displays the original content while preserving a link back to the source document, and may even trigger automatic copyright payment mechanisms.

This is entirely different from today's common "copy-and-paste" model. For example, if you transclude a passage from someone else's work into your own article, readers will know where that passage came from, and if the original text is updated, your referenced content may update along with it (depending on the implementation). You can embed fragments of other documents into your own while retaining attribution and copyright linkage to the original, rather than simply duplicating content. This mechanism inherently supports knowledge provenance, version tracking, and copyright protection.

Version tracking and provenance management: Every piece of content has a clear origin, version, and evolution history. In Xanadu's system, every modification to a document is recorded, forming a complete version tree. Users can view the historical evolution of any content and understand how it developed from its initial version step by step. This comprehensive version control is similar to the Git system used in software development today, but with a much broader scope, covering all types of documents and knowledge.

These features seemed like "over-engineering" in the era when the Web was primarily read by humans — ordinary users didn't need such rigorous citation networks, and simple click-to-navigate links were sufficient. This was a major reason Xanadu failed commercially.

Why AI Agents Change the Game

The article's core insight is this: AI agents' requirements for information networks align remarkably well with Xanadu's design philosophy.

The Capabilities and Limitations of Large Language Models

Large Language Models (LLMs) such as the GPT series and Claude are deep learning models that acquire language understanding and generation capabilities by training on massive text datasets. These models contain billions or even hundreds of billions of parameters and can perform tasks including translation, question answering, and writing. However, LLMs suffer from a serious issue known as "hallucination" — they sometimes generate information that sounds plausible but is actually inaccurate or entirely fabricated. This happens because models generate text based on statistical patterns rather than truly "understanding" or "fact-checking" information.

To mitigate this problem, the industry is developing techniques like Retrieval-Augmented Generation (RAG), which allows models to cite verifiable external sources when answering questions. When LLM-powered agents begin autonomously browsing, retrieving, and synthesizing web information, the pain points they face are precisely the shortcomings of today's Web:

Provenance and trustworthiness: Agents need to know exactly where information comes from in order to assess reliability and avoid hallucinations. Xanadu's transclusion mechanism naturally provides this traceability. This is exactly where Xanadu's transclusion can shine — if the network itself provides a structured, traceable citation system, agents can more easily verify information sources and reduce hallucinations.

Precise citation: When synthesizing multiple sources, agents need to cite specific passages rather than entire documents. Transclusion was built for exactly this purpose. When an AI agent needs to construct an argument or answer a complex question, it often needs to extract specific paragraphs or data points from multiple documents and combine them into a coherent response. Xanadu's transclusion mechanism makes this process structured and traceable, with each cited fragment retaining a link back to its original context.

Link stability: The knowledge graphs agents build depend on links remaining valid over time, and the Web's link rot is a fatal flaw for automated systems. AI systems often need to maintain and update knowledge bases over extended periods. If underlying data sources frequently go dead, the reliability of the entire knowledge system is severely compromised.

Bidirectional reachability: Agents need to understand the relational networks between pieces of information, and bidirectional links allow them to trace citation relationships in reverse. When researching a topic, an AI needs to know not only which resources a given document cites, but also which other documents cite that document. This kind of "influence map" is crucial for assessing the importance and credibility of information.

In other words, humans can tolerate vague, broken, and unsourced links because we have common sense and judgment to fill in the gaps. But for AI agents that require rigorous reasoning and precise citation, the structured, verifiable information network that Xanadu offers is precisely the underlying infrastructure they need most.

Diverse Voices in the Community Discussion

The Hacker News comment section presented a range of perspectives. Supporters argued that this viewpoint offers a new framework for re-examining Xanadu's legacy — a technology's value often depends on whether the era is ready to embrace it. Some commenters noted that many of Xanadu's ideas have been partially realized in today's Semantic Web, knowledge graph technologies, and similar efforts, just in different forms.

Other commenters were more cautious, pointing out that even if the ideas align, Xanadu's complex technical implementation and closed ecosystem remain formidable barriers. Critics argued that Xanadu's failure wasn't just about being ahead of its time — it was also because the design was overly idealistic, attempting to solve every problem at once while lacking the flexibility for incremental evolution. By contrast, the World Wide Web succeeded precisely because it started with a minimum viable solution and allowed the ecosystem to evolve naturally. Agents could perfectly well build their own citation layer on top of the existing Web, rather than depending on a system that never actually ran.

This discussion touches on a deeper question: Do we need to redesign the internet's underlying protocols for AI agents?

The Rise of Next-Generation Agent Infrastructure

As new agent-oriented standards emerge, the industry is addressing the problems Xanadu once tried to solve — through different paths.

The Significance of Model Context Protocol (MCP)

Model Context Protocol (MCP) is an open standard introduced by Anthropic in 2024, designed to enable AI agents to access various data sources and tools in a standardized way. Before MCP, every AI application had to write custom connection code for different data sources, leading to massive duplication of effort and interoperability problems. MCP defines a unified set of interfaces that allow agents to access file systems, databases, APIs, search engines, and other types of resources in the same way. The protocol includes data source registration, permission management, query execution, and other capabilities.

The emergence of MCP signals that the industry is beginning to take seriously the question of "designing infrastructure for AI." While MCP differs from Xanadu in its design philosophy, both attempt to solve a similar problem: how to enable automated systems to reliably and traceably access and integrate dispersed information resources. MCP represents a pragmatic path — building an agent-friendly protocol layer on top of the existing Web, rather than reinventing the entire network architecture.

Knowledge Graphs and Citation Provenance Tools

A knowledge graph is a technology that uses graph structures to represent and store knowledge, composed of entities (nodes) and relationships (edges). Each entity represents a concept or object (e.g., "Beijing," "capital," "China"), and relationships describe the connections between entities (e.g., "Beijing is the capital of China"). After Google launched its Knowledge Graph service in 2012, this technology saw widespread adoption in search engines, recommendation systems, question-answering systems, and other domains.

The advantage of knowledge graphs is that they provide structured, queryable knowledge representations that enable machines to perform logical reasoning rather than mere keyword matching. In AI agent scenarios, knowledge graphs can serve as external memory and a reasoning foundation, helping models understand complex relationships between entities and reducing hallucinations. In a sense, knowledge graphs realize part of Xanadu's bidirectional linking vision — relationships between pieces of information are explicit, traceable, and machine-readable. Unlike Xanadu, however, modern knowledge graphs are typically domain-specific and centrally constructed, rather than the universal literary universe Nelson envisioned.

The flourishing of various knowledge graphs and citation provenance tools shows that the industry is exploring agent-friendly information architectures from multiple angles. In a sense, the ghost of Xanadu is being reborn in these new technologies.

Validation Half a Century Late

Ted Nelson once said that what he designed was not a software product, but a philosophy about how human knowledge should be organized and connected. Half a century later, as AI agents become important "readers" of information networks, we are finally beginning to understand the true value of that philosophy.

The story offers a profound lesson for technology practitioners: A technology's failure doesn't necessarily mean it was wrong — it may simply mean its users hadn't arrived yet. What Xanadu was waiting for wasn't faster processors or greater bandwidth, but a class of agents that would truly need and appreciate its rigor. In the 1960s, when Nelson proposed these ideas, computers were room-filling behemoths, the internet had not yet been born, and AI was a concept from science fiction. His vision was so far ahead that it was nearly impossible for his contemporaries to understand or accept.

Of course, this doesn't mean Xanadu itself will be resurrected in its original form. What's more likely is that its core ideas — bidirectional links, transclusion, content permanence — will be integrated in new forms into the information infrastructure being rebuilt for the AI era. We probably won't see a system called "Xanadu" dominating the internet, but we will see more and more technologies adopting the principles Nelson proposed decades ago: traceability, transparency, permanence, and structured citation. These principles were luxuries in a human-centered web; in a network where agents are key participants, they become necessities.

History may not repeat itself, but it often rhymes. Xanadu's story reminds us that technological innovation requires not only vision but also the patience to wait for the right use cases and user base. In the AI era, many demands for rigor once dismissed as "over-engineering" may become the new standard. Nelson's dream may never be realized in the exact form he originally envisioned, but the intellectual seeds he planted are sprouting and growing in the next generation of information architecture.

Key Takeaways

• Proposed in 1960, the Xanadu project's features — bidirectional links, transclusion, and content permanence — were considered over-engineered in the era of human users
• AI agents require precise provenance tracking, stable links, and structured citation relationships — needs that align remarkably well with Xanadu's design philosophy
• The current World Wide Web architecture (unidirectional links, link rot, lack of citation traceability) poses serious obstacles for AI systems, undermining knowledge graph reliability and reasoning accuracy
• Emerging technologies like MCP and knowledge graphs are realizing parts of Xanadu's vision through different approaches, reconstructing information infrastructure for the age of agents
• A technology's success often depends on the maturity of its application scenarios and user base — visionary designs may need to wait decades before finding their true purpose