Can Lakebase Handle ML Real-Time Feature Serving? A Deep Dive into Latency and Concurrency

Introduction: The Convergence of Lakehouse Architecture and ML Serving Layers

As machine learning engineering shifts from batch processing to real-time inference, choosing the right data infrastructure has become increasingly critical. Recently, a developer on Reddit raised a highly representative question: Can Lakebase serve as a serving database for ML applications? Specifically, how does it perform in terms of latency and concurrency for real-time feature lookup and inference workloads?

This question touches on a core pain point in today's MLOps landscape: how to simultaneously meet both analytical (OLAP) and transactional (OLTP) requirements on a unified data platform. OLAP focuses on aggregation analysis over large-scale data, with typical operations including multi-table joins and group-by statistics, where query latency ranges from seconds to minutes. OLTP, on the other hand, serves online transaction scenarios dominated by high-frequency single-row reads and writes, demanding millisecond-level responses and strong consistency. For a long time, the industry has widely believed that these two types of workloads cannot coexist efficiently in the same system, giving rise to the classic architectural pattern of ETL pipelines that move data from transactional systems to analytical systems. In recent years, the HTAP (Hybrid Transactional/Analytical Processing) concept has emerged, with new databases like TiDB and CockroachDB attempting to support both workload types within a single system. Lakebase can be seen as Databricks' significant exploration in this direction.

What Is Lakebase: A Transactional Extension to Databricks' Lakehouse Architecture

Conceptual Positioning

Lakebase is a new database paradigm proposed within the Databricks ecosystem. Its core idea is to build Postgres-compatible transactional database capabilities directly on top of the Lakehouse architecture. It aims to bridge the traditional data lake's weakness of being "great at analytics but poor at low-latency point lookups."

To understand Lakebase's positioning, you first need to understand the Lakehouse architectural paradigm. The Lakehouse combines the flexibility of Data Lakes with the structured query capabilities of Data Warehouses. Traditional data lakes store raw data on object storage (such as S3 or ADLS), offering low cost but lacking transaction support and schema management. Data warehouses provide strong consistency and high-performance queries but come with high storage costs and limited scalability. The Lakehouse architecture introduces open table formats like Delta Lake and Apache Iceberg on top of object storage, enabling enterprise-grade features such as ACID transactions, Time Travel, and schema evolution while retaining the low cost and openness of data lakes. However, this architecture was originally designed primarily for analytical workloads, and support for low-latency point lookup scenarios has always been a gap waiting to be addressed — which is exactly the problem Lakebase aims to solve.

In simple terms, under traditional architectures, teams often need to maintain two separate systems: one for analytical workloads (a data warehouse or lakehouse) and another for online serving (an OLTP database like Postgres, Redis, etc.). Data must be synchronized back and forth between the two, adding operational complexity and introducing data consistency risks. Lakebase's goal is to combine "disaggregated storage and compute + Postgres engine" so that the same data can be read by analytics engines and accessed with low latency by online applications.

Why ML Real-Time Inference Scenarios Pay Special Attention to Lakebase

For ML inference services, the most typical requirement is feature lookup: when an inference request arrives, the model needs to pull the corresponding feature vector for that user or entity from the database in real time. In the MLOps toolchain, this capability is typically provided by a Feature Store. Open-source and commercial feature stores like Feast, Tecton, and Hopsworks manage the feature lifecycle by providing unified feature definitions along with a two-tier architecture of offline (batch) storage and online (low-latency) storage. The online store typically uses key-value databases like Redis or DynamoDB, offering sub-millisecond point lookup performance; the offline store is based on data lakes or data warehouses for batch feature backfill and training dataset construction. Lakebase attempts to cover both tiers within a single system, but whether it can match the performance of dedicated key-value stores at the online tier remains a key question.

This type of feature lookup operation has the following characteristics:

• Primarily point lookups: Typically single-row or small-row queries based on primary keys
• High concurrency: Production services may reach thousands of QPS or higher
• Latency-sensitive: Feature lookups often sit on the critical path of the inference pipeline, where even a few milliseconds of additional latency can impact overall SLA

The SLA (Service Level Agreement) mentioned here typically uses percentile latency as the key metric in online services — for example, "P99 latency must not exceed 50ms." P99 latency means that 99% of requests complete within that threshold. Compared to average latency, P99 better reflects the actual user experience because in microservice architectures, a single user request may chain-call multiple services, and tail latency at each service gets amplified. Even if model inference itself is fast, an unstable feature service can cause frequent SLA violations.

This is the core concern of the original poster: can Lakebase match a purpose-tuned Postgres or a dedicated Feature Store across these three dimensions?

Core Challenges of Real-Time Feature Lookup

Latency: Disaggregated Storage Architecture Sets the Floor

Lakebase is built on a disaggregated storage-compute lakehouse architecture, which brings elastic scaling and cost advantages but may also introduce additional latency overhead. Disaggregated Storage and Compute is one of the core design principles of cloud-native database architectures. It splits the tightly coupled storage and compute layers of traditional databases into independent components: the storage layer typically relies on object storage (like AWS S3) or distributed file systems, while the compute layer consists of stateless query engine nodes. The advantage of this architecture is that storage and compute can scale independently — storage is virtually unlimited and pay-per-use, while compute resources can elastically scale up or even down to zero on demand. Major cloud data warehouses like Snowflake, BigQuery, and Redshift Serverless all adopt this architecture. However, the tradeoff of disaggregated storage is that data access must cross network boundaries — even within the same availability zone, network round-trip latency is one to two orders of magnitude higher than local NVMe SSDs.

Compared to traditional Postgres with local disk or in-memory caching, data access through the object storage layer is often slower in cold-read scenarios. For ML feature serving, the key lies in hot data caching strategies. If Lakebase has a well-designed memory/SSD caching layer that keeps frequently accessed features on low-latency media, its P99 latency could potentially stay within an acceptable range. But if cache hit rates are poor, latency spikes from cold starts or tail queries could become a production hazard.

Concurrency: Connection Management and Auto-Scaling

PostgreSQL uses a "process-per-connection" model, where each time a client establishes a connection, the Postgres master process forks an independent backend process to handle all requests for that connection. The advantage of this model is strong isolation and simple implementation, but it exposes clear bottlenecks under high concurrency: each process consumes roughly 5-10MB of memory, so hundreds of connections can consume several GB; context switching among numerous processes significantly increases CPU overhead; and contention for file descriptors and shared memory locks intensifies. In real production environments, when connections exceed a few hundred, Postgres throughput not only stops growing but may actually decline.

This is why connection pooling middleware like PgBouncer and Pgpool-II emerged — they maintain a connection pool between the application and database, serving large numbers of client requests by reusing a small number of persistent connections. Next-generation Postgres-compatible databases (such as Amazon Aurora, Neon, Supabase, etc.) typically redesign connection management at the architectural level to better handle the high concurrency demands of cloud-native and microservice scenarios.

Whether Lakebase, as a Postgres-compatible service, natively solves this problem is an important factor in evaluating its concurrency capabilities. If Lakebase employs modern connection management and auto-scaling mechanisms, it could theoretically handle burst traffic from inference services more effectively. Otherwise, if it's still constrained by the classic Postgres connection model, teams will need to do additional capacity planning.

Potential Pitfalls of Lakebase Compared to Traditional Postgres

The original poster specifically asked about "gotchas compared to regular Postgres." While the community hasn't yet provided extensive real-world feedback, based on common patterns of similar architectures, several aspects deserve attention.

The Consistency-Latency Tradeoff

Lakehouse architectures typically require commit and snapshot processes after writes, so there may be a delay between when data is written and when it becomes queryable. For feature update scenarios that require "read-your-writes" semantics, this must be carefully verified — otherwise, inference may read stale features.

This issue is closely related to "Training-Serving Consistency." Training-serving consistency requires that the features used during model training and those retrieved during online inference are completely identical in definition, computation logic, and data source. If training uses features generated by an offline batch pipeline while inference computes features through a different real-time pipeline, even if the logic is "the same," subtle differences in data processing timing, precision, aggregation windows, and other details can lead to so-called "training-serving skew" — where the model shows excellent offline evaluation metrics but poor online performance. Google identified such issues as one of the most common sources of technical debt in ML systems in their classic paper Machine Learning: The High-Interest Credit Card of Technical Debt. Lakebase reduces the risk of this skew at the architectural level by allowing training and serving to share the same data store, which is one of its most attractive features for ML teams. But the premise is that written data can be read in a timely manner; otherwise, the consistency promise is significantly weakened.

Cold Starts and Cache Warming

As mentioned earlier, disaggregated storage architectures are highly dependent on caching. During service restarts, scale-out events, or sudden changes in access patterns, insufficient cache warming can cause latency spikes in the short term. Production environments need well-designed warming and monitoring mechanisms.

Cost Model Differences

Unlike fixed-instance Postgres, lakehouse-style services often use usage-based pricing. In high-concurrency, continuous-query ML serving scenarios, cost models need to be evaluated upfront to avoid situations where "performance meets targets but the bill spirals out of control."

Selection Recommendations: When to Consider Lakebase for ML Feature Serving

Combining community discussions and architectural characteristics, here are some practical selection guidelines:

Suitable scenarios:

• The team is already heavily invested in the Databricks/Lakehouse ecosystem and wants to reduce data synchronization pipelines
• Feature data and training data share the same source, pursuing "training-serving consistency" — meaning the training pipeline and online inference use the same feature data, fundamentally avoiding training-serving skew caused by data sync delays or computation logic differences
• Latency requirements fall in the moderate range (tens of milliseconds are acceptable), rather than demanding single-digit millisecond performance

Scenarios requiring caution:

• Strict P99 latency requirements (e.g., single-digit milliseconds) for high-frequency trading-style inference
• An existing mature dedicated feature store (e.g., Redis + Feature Store) that's working well
• The team lacks experience with lakehouse architecture caching and tuning

Conclusion

Lakebase represents an important direction in data infrastructure "unification" — using a single system to handle both analytical and serving workloads, which is extremely attractive for simplifying MLOps architecture. But as this Reddit developer's question reveals, architectural elegance doesn't automatically translate to production-ready answers.

Latency, concurrency, consistency, and cost — these four dimensions still need to be repeatedly validated under real workloads. For teams currently evaluating Lakebase, we recommend first running benchmarks with real feature query patterns, focusing on P99 latency and cache hit performance, before deciding whether to incorporate it into critical inference pipelines. In the current absence of large-scale production case studies, "validate incrementally, progress gradually" remains the safest strategy.