Perplexity's In-House CobbleDB: 2 Engineers + Hundreds of AI Agents Built Search Infrastructure in 2 Months

Perplexity Reveals CobbleDB Technical Details

Perplexity recently announced the publication of technical research on its in-house key-value database, CobbleDB. This database serves as the backbone of Perplexity's search product, handling the storage and retrieval of web content that powers its AI search experience.

According to Perplexity's official disclosure, CobbleDB's core infrastructure was built by a team of just two engineers working alongside hundreds of proactive, always-on AI agents — and the whole thing came together in two months. That development model, arguably, deserves more attention from the tech industry than the database product itself.

Why a Search Engine Needs Its Own Key-Value Database

A key-value database is a type of NoSQL database that stores data as key-value pairs. Known for efficient read/write performance and strong horizontal scalability, it's widely used in scenarios that demand high throughput and low latency.

For an AI search product like Perplexity, the system must crawl, store, and rapidly retrieve massive amounts of web content. General-purpose database solutions often struggle to strike the right balance between cost, latency, and scale. Building CobbleDB in-house means Perplexity can deeply optimize for the specific access patterns of its content serving layer — such as read-heavy workloads over web documents and large-scale parallel retrieval — giving it better performance and cost control on critical paths.

For search products, the efficiency of the content serving layer directly affects response speed and operating costs, making ownership of the underlying database a strategically significant decision.

---

In the search engine space, key-value databases are typically used to store raw web page content, index metadata, or crawl snapshots. Popular open-source options include RocksDB (developed by Facebook, built on an LSM-tree structure with extremely high write throughput) and LevelDB (developed by Google), and many large-scale systems build custom abstraction layers on top of them. Perplexity's choice to build from scratch rather than adopt one of these mature solutions likely signals some unusual requirements — such as vector proximity query optimizations tailored for AI retrieval pipelines, specific compression strategies, or deep adaptation to a cloud-native disaggregated storage-compute architecture. The trade-off is that building in-house means independently shouldering correctness verification, failure recovery, and long-term maintenance — which is precisely why only a handful of companies like Google and Meta have historically gone down this path.

---

What the "2 Engineers + Hundreds of AI Agents" Model Really Means

The most discussion-worthy aspect of this announcement is the development methodology itself: two engineers leading the effort, coordinating with hundreds of "always-on" AI agents to build core infrastructure.

AI Agents Moving from Support Role to Primary Contributor

Over the past year, AI coding tools have largely played the role of "pair programming assistants" — helping developers autocomplete code or generate snippets on demand. What Perplexity describes goes a step further. These AI agents aren't just passively responding; they're described as "proactive and always-on," functioning as orchestratable productivity units within the engineering team.

If this holds up, it represents a potential shift in how software engineering organizations are structured: a small number of senior engineers handling architectural decisions and direction, while large numbers of AI agents take on the heavy lifting of implementation, testing, and iteration. Systems-level software like databases — where correctness and concurrency safety requirements are extremely high — has traditionally been seen as a domain where AI struggles to contribute meaningfully. That's precisely what makes Perplexity's case a notable signal.

---

The industry has yet to converge on a single definition of "AI agent," but in software engineering contexts the term generally refers to automated systems capable of independently planning subtasks, invoking tools (such as code execution, file I/O, and test runners), and iterating based on results. Representative frameworks include Devin, SWE-agent, and various multi-agent orchestration systems built on large language models. The key difference from traditional code completion tools like GitHub Copilot is that agents can operate across multiple steps continuously, well beyond the boundary of a single prompt. The description of "hundreds of always-on agents" suggests Perplexity may be running a large-scale parallel agent cluster — with each agent handling an independent subtask (e.g., writing unit tests, implementing a specific module, fixing lint errors) while a small number of engineers handle top-level task assignment and result review. In this model, the engineer's core value shifts from "writing code" to "defining correctness standards and acceptance criteria."

---

Where Skepticism Is Warranted

It's worth noting that the information currently available comes primarily from Perplexity's own promotional posts, and there has been no independent third-party verification of the development process, code quality, or AI contribution ratio. Exactly what portion of the "core infrastructure" was completed in "two months," and how much actual decision-making the AI agents performed, remains to be clarified once the full technical research report is published. As part of a marketing narrative, the efficiency figures cited should be interpreted with appropriate caution alongside whatever technical details are publicly released.

Implications for the Industry

The CobbleDB story sits at the intersection of two converging trends: AI companies increasingly choosing to build critical infrastructure in-house to control performance and costs, and AI agents being brought into their own product development workflows — creating a feedback loop of "using AI to build AI."

For engineering teams, this points to an organizational experiment worth watching: using agents at scale to handle implementation-layer work and compress delivery timelines. Equally important, however, are the complementary mechanisms for validation, review, and reliability assurance — especially in domains like databases where the margin for error is razor-thin.

Perplexity has indicated it will publish the associated research. The complete technical documentation will be the real test of how substantive this model truly is. Practitioners who are interested should keep an eye on their forthcoming publications for more concrete engineering details.