Training a 4B Small Model to Generate Query Plans That Beat Postgres by 81%

Rethinking Database Query Optimization with Small Models

The query optimizer is one of the most critical and complex components in a relational database. It determines how a SQL statement gets executed — which table to scan first, which index to use, and in what order to perform JOINs. For the same query, different execution plans can differ in performance by an order of magnitude or more. Mature databases like Postgres rely on heuristic optimizers built on cost models, which estimate the cost of each execution path using statistical information and then select the lowest-estimated-cost option.

This approach has worked for decades, but it has inherent limitations: the statistical estimates that cost models depend on are frequently off — especially in scenarios involving multi-table joins, skewed data distributions, or correlated columns — causing the optimizer to make poor choices. A recent project that caught the attention of the Hacker News community proposes an alternative: using a small model with just 4B parameters to generate query plans, claiming the resulting execution plans are 81% faster than Postgres's native optimizer.

Why a 4B Model Can Outperform a Traditional Optimizer

The core idea behind that number is transforming query optimization from a "rule-and-statistics-based search" problem into a "learning" problem. Estimation errors in traditional optimizers compound across multiple layers of JOINs, whereas a machine learning model can learn directly from large volumes of real execution data — figuring out which plans actually perform better and bypassing the estimation bias inherent in cost models.

The choice of model scale is worth noting. 4B parameters is considered a "small model" in today's large model landscape, which means it has a chance to complete inference within reasonable latency and resource budgets — a critical constraint for database systems. Query optimization must complete in milliseconds; spending hundreds of milliseconds on model inference just to produce a slightly better plan would be counterproductive for most online query workloads. Choosing 4B rather than a much larger model is itself a deliberate tradeoff between "optimization gains" and "inference overhead."

The "81% faster" metric deserves careful interpretation. It typically refers to the improvement in execution time for model-generated plans versus Postgres's default plans on a specific benchmark set (such as complex analytical queries). This kind of gain is much easier to achieve on analytical (OLAP) workloads, where queries are more complex and the optimization space is larger. For simple point lookups or transactional (OLTP) workloads, Postgres's native optimizer is often already good enough and there's limited room for improvement.

The root cause of estimation errors in traditional optimizers lies in the cardinality estimation problem — predicting how many rows a query condition will return. Single-table cardinality estimation can rely on histogram statistics, but for multi-table JOINs, optimizers typically assume column independence and multiply individual selectivity values to estimate result set sizes. In reality, columns are often correlated (e.g., city and zip code are highly correlated), and this independence assumption can cause estimates to be off by several orders of magnitude. Incorrect cardinality estimates cause the optimizer to misjudge JOIN costs, leading to wrong JOIN orderings or join algorithms (e.g., choosing Nested Loop over Hash Join), which causes severe performance degradation at scale. This is precisely why machine learning approaches are so promising — they can implicitly learn the internal patterns of data distributions from large volumes of real execution results, without needing to explicitly model inter-column correlations.

Learned Query Optimization Is Not a New Concept

Using machine learning to improve database query optimization has been an active academic research direction for years. Systems like MIT's Neo, Microsoft's learned cardinality estimation for SQL Server, and Bao have all explored similar ideas. These efforts broadly fall into a few categories: using models to improve cardinality estimation, using reinforcement learning to guide JOIN order search, or directly generating execution plans end-to-end.

The significance of this 4B model project is that it brings large language model-era training methods into this field. Compared to earlier learned optimizers that required carefully designed feature engineering, using a unified neural network model to learn directly from queries and data is simpler from an engineering standpoint and easier to adapt to new workloads. That said, the community discussion raised several critical questions worth considering for anyone thinking about real-world deployment.

The representative systems mentioned here each take a different approach, and it's worth briefly distinguishing them. Neo (Neural-Enhanced Optimizer, MIT 2019) takes an end-to-end approach, directly mapping query trees to execution plans through imitation learning and reinforcement learning. Bao (Bandit Optimizer, MIT 2021) is more conservative: rather than generating plans from scratch, it provides different hints to the traditional optimizer (e.g., disabling certain JOIN algorithms), lets the optimizer search within a constrained space, and uses a bandit algorithm to learn from execution feedback which hint combinations work best — a design that naturally preserves the traditional optimizer's fallback capability. Microsoft's learned cardinality estimation in SQL Server focuses on a single problem: replacing histograms with neural networks for more accurate row count predictions, improving the input quality to the cost model rather than replacing the entire optimization pipeline. In contrast, using a large language model-style architecture with 4B parameters to directly generate plans represents a much more aggressive approach.

Key Questions That Must Be Answered Before Deployment

First, generalization capability. A model may perform excellently on query patterns covered by its training data, but how does it behave when faced with unseen schemas, data distributions, or query structures? The reality of databases is that schemas and data continuously evolve. A "frozen" model will likely degrade over time, requiring mechanisms to detect this and trigger retraining.

Second, worst-case reliability. Traditional optimizers, even when suboptimal, are predictable in their behavior. A learned model that occasionally generates a catastrophically bad plan could cause critical queries to time out. Production environments typically need a "safety net" — for example, using the model as an advisor that compares its plan against the traditional optimizer's and picks the better one, or setting up fallback mechanisms — rather than a full replacement.

Third, interpretability and debugging. DBAs troubleshooting slow queries rely on EXPLAIN output to understand the optimizer's decision logic. When plans are generated by a neural network, debugging why a particular plan was selected becomes a new operational challenge.

Fourth, training cost and data acquisition. Training a high-quality model requires executing large numbers of queries in real or simulated environments to collect feedback signals. This data collection process itself carries significant cost and may also interfere with production systems.

Distribution shift is one of the long-term challenges these systems face. A database's data volume, data distribution, and query patterns continuously evolve with business growth — holiday traffic spikes, query pattern changes from new feature launches, and index selectivity changes from data growth can all cause the "optimal plan" learned during training to no longer be optimal. Engineering teams need to design continuous monitoring mechanisms: comparing model-recommended plans against actual execution results and triggering incremental or full retraining when systematic deviations are detected. Additionally, the cold start problem is equally thorny — a newly deployed database has no historical execution data, and a model can't be trained on an empty dataset. Typically, you need to first accumulate enough execution samples using the traditional optimizer before switching to a learned approach. This means learned optimizers are fundamentally "living systems" that require continuous maintenance, not static components you deploy once and forget.

A Directional Signal

Setting aside the debate over a single benchmark number, the greater value of this project is as a directional signal: as model inference costs decline and small model capabilities improve, embedding learning capabilities into the core components of database systems is becoming increasingly feasible. Query optimization is just the starting point — caching strategies, index recommendations, and resource scheduling all present similar opportunities.

For practical engineering teams, the more pragmatic approach right now may not be replacing a mature optimizer wholesale with a model, but rather using the model as an enhancement layer — letting the model identify suboptimal decisions in the plans produced by the traditional optimizer and suggest improvements. This captures the benefits of the learned approach while preserving the reliability floor of the traditional method. Once the model's robustness and generalization capabilities are thoroughly validated, deeper integration will follow naturally.