Tabular Foundation Models: A New Paradigm for Data Science Agents Handling Structured Data

Tabular foundation models fill the gap in LLMs for structured prediction through pretraining plus in-context learning.
The article introduces a course titled "Tabular Foundation Models for Data Science Agents," created in collaboration with Prior Labs, which systematically explains tabular foundation models. Traditional machine learning requires training and tuning a separate model for each dataset, whereas tabular foundation models are pretrained on millions of tabular tasks and, when faced with new data, directly produce predictions through in-context learning, eliminating feature engineering and hyperparameter tuning. The course covers hands-on classification and regression (using TapiaFN), sub-second real-time inference based on KV cache, dual-layer global and individual interpretability, and how to integrate TapiaFN into an AI agent through MCP and skill: the LLM determines the task type and orchestrates, while the tabular model performs prediction. The article argues that this division-of-labor architecture is more valuable than simply stacking model capabilities.
When LLMs Meet Tables: An Overlooked Blind Spot
Large language models have reshaped the way we handle unstructured data such as text and code, but they are not good at tabular data. Whether it's predicting customer churn, demand forecasting, or assessing patient risk, these kinds of structured prediction problems are precisely where LLMs fall short. According to the video, a course titled "Tabular Foundation Models for Data Science Agents" was created in collaboration with Prior Labs, with instructors including Frank Kutter, Taneh Jellik, and Elliot Kalfin, systematically explaining how tabular foundation models are changing this situation.

Traditional machine learning requires training and tuning a model for each dataset—a lengthy process that is hard to reuse. The core difference of tabular foundation models is that they are pretrained on millions of different tabular prediction tasks. When faced with a new dataset, they learn patterns directly from the examples in the table through in-context learning, without the need to rebuild models. This means the data science workflow is greatly simplified.
How Tabular Foundation Models Work
In-Context Learning Replaces Per-Dataset Training
Tabular foundation models are not specialized models trained for a single task, but pretrained models with cross-task generalization capabilities. Given a new table, they can read the examples in context and produce predictions, eliminating the repetitive work of feature engineering and hyperparameter tuning. The course explains in detail why LLMs are not suitable for this type of problem, and to what extent the workflow can be simplified after migrating from traditional machine learning to tabular foundation models.

In-Context Learning (ICL) originates from the working mechanism of large models such as GPT-3: model parameters are not updated, and the model can complete a task based solely on a few examples given in the prompt. Tabular foundation models transfer this idea to structured prediction—placing several rows from the training set as "examples" into the context, and the rows to be predicted as "queries," with the model performing "analogical reasoning" during forward computation.
This is fundamentally different from traditional machine learning: the traditional pipeline requires fitting parameters, saving model artifacts, and deploying inference services; ICL compresses "learning" into a single forward pass, with the model itself remaining frozen. The cost is that each prediction must carry examples, and the longer the context, the higher the inference cost—so the selection and ordering of examples affects performance. For scenarios with extremely large data volumes or high feature dimensions, sampling and retrieval strategies are still needed.
Hands-On Classification and Regression
In the course, learners will personally use TapiaFN to complete a classification task and a regression task. The value of this hands-on segment is that it grounds the abstract concept of "pretraining + in-context learning" in concrete datasets, allowing learners to intuitively experience the difference from traditional modeling workflows.

Real-Time Inference and Interpretability
In real deployments, predictions often need to be returned in real time. The course explains how KV cache reduces latency to sub-second levels to meet the demands of real-time prediction. Interpretability is also covered: learners will examine which features influence the model at a global level, and what drives a specific individual prediction. For high-risk scenarios such as risk control and healthcare, this dual-layer "global + individual" explanatory capability is a prerequisite for deployment.

KV cache (key-value cache) is a core technique for accelerating Transformer inference. During autoregressive generation, each layer of attention needs the Key and Value vectors of all previous tokens; without caching, every token generated requires recomputing the entire history, with complexity growing quadratically with sequence length. KV cache saves these intermediate tensors, so a new token only needs to compute itself and reuse the historical cache, reducing repeated computation to a single pass.
In tabular foundation models, the large number of example rows in the context forms a very long prefix, and this portion is fixed within a single prediction request—making it ideal for residing in the KV cache. Cache reuse allows the initial loading of examples to be separated from subsequent queries, thereby reducing single-prediction latency to sub-second levels. In actual deployment, cache capacity and multi-tenant isolation must also be considered—different datasets correspond to different caches, which need to be managed by session or task.
Collaboration with AI Agents: Division of Labor, Not Replacement
At the end of the course, TapiaFN is integrated into an AI agent. The LLM is responsible for reasoning about whether the current task is a prediction task, and if so, hands it over to TapiaFN for processing. The TapiaFN MCP provides the agent with access channels to models and prediction tools, while the skill tells the agent when and how to use these tools. This design reflects a pragmatic architectural approach: the LLM does what it is good at—task judgment and orchestration—and the tabular foundation model does what it is good at—structured prediction—each fulfilling its own role.
MCP (Model Context Protocol) is an open protocol for connecting LLM applications to external tools and data sources in a standardized way. It abstracts "how a model discovers and calls a tool" into a unified interface: the server exposes a list of tools and parameter schemas, and the client (agent) decides how to call them accordingly. Compared with writing custom integration code for each tool, MCP makes tool integration reusable.
The skill mentioned in the article is a higher-level semantic description that tells the agent under what conditions and in what order to use a certain tool. The relationship between the two can be understood as: MCP solves the connectivity problem of "can it be called," while skill solves the decision problem of "should it be called, and how." This layering allows tabular foundation models to be integrated into any MCP-compatible agent framework without being tied to a specific implementation.
The Next Stop for Structured Data
Tabular foundation models offer a new path for the large number of classification, regression, and prediction tasks in science and industry. Their significance lies not in replacing everything in traditional machine learning, but in changing the cost structure of "training from scratch for every dataset." For teams building data science agents, understanding the boundary between LLMs and tabular models may be more valuable than simply stacking model capabilities.
Related articles

Gemini 4 Neon Released: Beats Opus 5.5 on Multiple Benchmarks, Google Returns to the Top Tier
Google DeepMind releases Gemini 4 Neon: 61 on the Artificial Analysis intelligence index (3rd) and 1st on the coding index, beating Opus 5.5 on multiple benchmarks. $3/$15 per million tokens; available in the Gemini API and AI Studio.

Gemini 4 Neon Review: Benchmark Scores Rival Carbon, API Cost-Performance Analysis
Full review of Google Gemini 4 Neon: benchmark scores rival Carbon, API priced at 2x Hydrogen, yet Antigravity quota consumption is only 1x (vs Carbon's 3x). Real tests show it replaces 90% of Carbon use cases. Includes channel restrictions, ban risks, and combination strategy analysis.

A Billion-Dollar Deal: A Turning Point for the U.S. Battery Market?
Sila and Panasonic Energy sign a billion-dollar silicon-carbon anode supply deal. This article analyzes its impact on the U.S. next-gen battery market, supply chain localization, and commercialization prospects.