How Agentic Apps Are Reshaping the Data Stack: Why Databricks Is Betting on Postgres and LTAP

The New Pressure Agentic Apps Place on the Data Stack

With the rise of agentic applications, the traditional data stack is facing unprecedented challenges. These applications don't simply read from and write to databases — they must continuously maintain agent memory, application state, and historical data throughout execution, with frequent interactions across all three. Databricks co-founder and Chief Architect Reynold Xin (@rxin) noted in a recent keynote that it is precisely this new class of workload that is forcing the industry to rethink the foundational design of data processing architectures.

The way AI agents operate is fundamentally different from traditional applications. When an AI Agent executes a task, it needs to read its own memory in real time, retrieve historical context, update current state, and potentially trigger analytical queries against existing enterprise data — all at once. This tight interweaving of transactional and analytical processing is exactly where classic data architectures struggle most.

Why Postgres

One of Xin's core arguments is that Postgres will become the cornerstone of this new data processing architecture. The reasoning is clear. Postgres is not only a mature, stable, and widely adopted open-source relational database — more importantly, it has accumulated a vast ecosystem of drivers, extensions, and ORM tools. For enterprises, migration cost and compatibility are typically the biggest barriers to adopting new architectures.

Building on Postgres means the new architecture can maintain full compatibility with the existing Postgres ecosystem. Developers don't need to swap out drivers, rewrite ORM layers, or give up their existing extensions. This "backward-compatible" design philosophy lowers the barrier for enterprises to adopt agentic data capabilities and makes Databricks' approach far easier to integrate into existing technology stacks.

Postgres' emergence as a de facto "invisible standard" in AI infrastructure is closely tied to the evolution of the vector database ecosystem. Extensions like pgvector enable Postgres to natively store and retrieve high-dimensional vectors — which is the core operation for agent memory systems, where conversation history and knowledge fragments are encoded as vectors for semantic retrieval. Meanwhile, Postgres' logical replication protocol is widely supported by cloud services and data tools, making it a natural hub for data movement. In agentic applications, Postgres protocol compatibility means developers can use the same connection strings and drivers to seamlessly bridge local development environments and cloud production systems, dramatically reducing the friction of going from prototype to production.

LTAP: Letting Transactions and Analytics Read from the Same Data

The most noteworthy technical concept from this session is LTAP — enabling transactional and analytical processing to read from the same copy of data. For a long time, enterprises have relied on architectures that separate OLTP (Online Transactional Processing) and OLAP (Online Analytical Processing), requiring data to flow between the two systems via ETL or sync pipelines.

This separated architecture reveals clear shortcomings in agentic scenarios. Xin specifically highlighted the problem of sync intervals: in an agent memory loop, if transactional data must wait for synchronization before it can be read by analytical queries, that latency becomes a bottleneck for the entire loop. Agents need to make decisions based on the latest state, and stale data directly degrades their performance.

The value of LTAP lies in eliminating this bottleneck. When transactions and analytics read from the same data, there is no sync delay — agents can always operate on real-time, consistent data. For Agent applications that require high-frequency memory reads and writes, this is a fundamental architectural improvement.

LTAP (Lakehouse Transaction and Analytics Processing) is an extension and upgrade of the traditional HTAP (Hybrid Transactional/Analytical Processing) concept. The idea behind HTAP was proposed by Gartner around 2014, with the same goal of enabling a single system to handle both transactional and analytical workloads — representative products include TiDB and SingleStore. What sets LTAP apart is that it places this capability within the context of Lakehouse architecture — where the underlying storage is typically object storage in open formats (such as Delta Lake), rather than traditional row or column stores. This means data can support low-latency writes while being directly scanned by analytical engines, without maintaining two separate physical storage layers. For agentic scenarios, this is particularly critical: memory and state entries written by an Agent become visible to analytical queries within milliseconds, fundamentally eliminating the data freshness issues introduced by traditional ETL pipelines.

Unified Permissions: Memory, State, and Historical Data

Beyond performance, Xin also raised a dimension that is often overlooked but critically important — permission governance. In agentic applications, agent memory, application state, and historical data are often scattered across different systems, each maintaining its own independent access control policies, creating security risks and management complexity.

The goal of the new architecture is to bring all three categories of data under a unified permission framework. Unified permission management not only simplifies governance but also better meets the strict compliance and security requirements of enterprise-grade applications. For organizations handling sensitive data, the ability to cover all of an agent's data touchpoints with a consistent permission model is a prerequisite for deploying to production.

The complexity of permission governance for agentic applications stems from the heterogeneous nature of their data touchpoints. A typical Agent system may simultaneously involve: in-memory caches for short-term context (e.g., Redis), vector databases for long-term memory, relational databases for business state, and data lakes for historical logs. Each of these systems has its own authentication mechanisms and access control models. When an agent operates across systems, security auditors have virtually no way to reconstruct a complete data access trail from a single view. In regulated industries such as finance and healthcare, this fragmented permission model may directly violate the Principle of Least Privilege and audit trail requirements. Unifying all three data categories under a single permission framework is, in essence, extending the metadata governance capabilities of tools like Unity Catalog into the agent's runtime data domain.

Observations on the Evolution of Data Architecture

This session reflects a broader industry trend: AI Agents are becoming the new force driving transformation in data infrastructure. In the past, the evolution of the data stack was largely centered on BI analytics and big data processing. Today, agents' demanding requirements for real-time availability, consistency, and unified governance are forcing the convergence of transactional and analytical systems.

Databricks' choice to use Postgres as the foundation and bridge transactions and analytics via LTAP reflects a pragmatic approach — meeting the new demands of agentic applications while respecting enterprises' existing technology investments. This conversation about data architecture for agentic applications will be explored further through live demos and enterprise deployment patterns at an event running online November 10–12.

Note: This article is based on activity previews published by Databricks on social media. Specific technical details are subject to official documentation.