[KongchangAI]
Concepttabular foundation model

Tabular Foundation Model

A tabular foundation model is a class of model pretrained on large, heterogeneous tabular datasets that can learn relationships among columns, cell semantics, and table structure, enabling prediction without manual feature engineering. Representative works include Kumo and Nori.

Timeline (last 90 days)

Nov 11

Tabular foundation models are pretrained on millions of different tabular prediction tasks; when facing a new dataset, they learn patterns directly from examples in the table through in-context learning (ICL) without rebuilding the model.

Unverified50%
Nov 11

When actually deploying tabular foundation models, the capacity of the KV cache and multi-tenant isolation must be considered; different datasets correspond to different caches, which need to be managed by session or task.

Unverified50%
Nov 11

In tabular foundation models, KV cache reuse can separate the initial loading of examples from subsequent queries, thereby reducing single-prediction latency to sub-second levels.

Unverified50%
Nov 11

The cost of ICL is that every prediction must carry examples, and the longer the context, the higher the inference cost; the selection and ordering of examples affect performance; scenarios with extremely large data volumes or very high feature dimensions still require sampling and retrieval strategies.

Unverified50%
Nov 10

Competition in tabular foundation models is shifting from model architecture to pretraining data recipes, and the scale and diversity of synthetic data have become the core variables that create gaps.

Unverified50%
Nov 10

Tabular foundation models typically need to serialize a row of samples into a token sequence and adopt column-aware encoding to handle heterogeneous schemas.

Unverified50%
Nov 10

Benchmark datasets in materials science are typically limited in scale (tens to hundreds of samples) and differ in distribution from the large-scale synthetic data that TFMs encounter during pretraining; the heterogeneity of materials data may fall outside the pretraining distribution range.

Unverified50%
Nov 10

TFMs are based on the Transformer architecture, and how their inference overhead on large datasets compares with GPs needs to be judged in light of the specific scale; the paper's abstract does not address the inference cost dimension, while in actual materials discovery workflows the surrogate model needs to be rapidly updated and make predictions in each iteration round.

Unverified50%
Nov 10

Acquisition functions in Bayesian optimization (such as EI, UCB, and Thompson sampling) directly rely on the predicted mean and uncertainty output by the surrogate model; if the uncertainty calibration of TFMs is poor, the acquisition function may bias toward over-exploration or premature convergence.

Unverified50%
Nov 10

A recent study from arXiv proposes using tabular foundation models (TFMs) as surrogate models for Bayesian optimization, requiring no kernel function selection and no additional training for active learning tasks.

Unverified50%

9 more timeline events

All Facts (19)

Unverified

When actually deploying tabular foundation models, the capacity of the KV cache and multi-tenant isolation must be considered; different datasets correspond to different caches, which need to be managed by session or task.

50%
Unverified

In tabular foundation models, KV cache reuse can separate the initial loading of examples from subsequent queries, thereby reducing single-prediction latency to sub-second levels.

50%
Unverified

The cost of ICL is that every prediction must carry examples, and the longer the context, the higher the inference cost; the selection and ordering of examples affect performance; scenarios with extremely large data volumes or very high feature dimensions still require sampling and retrieval strategies.

50%
Unverified

Tabular foundation models are pretrained on millions of different tabular prediction tasks; when facing a new dataset, they learn patterns directly from examples in the table through in-context learning (ICL) without rebuilding the model.

50%
Unverified

Tabular foundation models typically need to serialize a row of samples into a token sequence and adopt column-aware encoding to handle heterogeneous schemas.

50%
Unverified

Competition in tabular foundation models is shifting from model architecture to pretraining data recipes, and the scale and diversity of synthetic data have become the core variables that create gaps.

50%
Unverified

Acquisition functions in Bayesian optimization (such as EI, UCB, and Thompson sampling) directly rely on the predicted mean and uncertainty output by the surrogate model; if the uncertainty calibration of TFMs is poor, the acquisition function may bias toward over-exploration or premature convergence.

50%
Unverified

The researchers compared TFMs with baseline models such as Gaussian processes, random forests, and CatBoost on two standard materials science benchmark datasets.

50%
Unverified

The study's core finding is that on both benchmarks, TFMs demonstrated highly competitive performance without requiring task-specific training or GP kernel function selection.

50%
Unverified

The researchers' wording is that TFMs are "highly competitive" rather than comprehensively surpassing Gaussian processes; GP's theoretical guarantees for uncertainty quantification and mature toolchain remain its advantages.

50%
Unverified

Benchmark datasets in materials science are typically limited in scale (tens to hundreds of samples) and differ in distribution from the large-scale synthetic data that TFMs encounter during pretraining; the heterogeneity of materials data may fall outside the pretraining distribution range.

50%
Unverified

TFMs are based on the Transformer architecture, and how their inference overhead on large datasets compares with GPs needs to be judged in light of the specific scale; the paper's abstract does not address the inference cost dimension, while in actual materials discovery workflows the surrogate model needs to be rapidly updated and make predictions in each iteration round.

50%
Unverified

Tabular foundation models are sensitive to column semantics and data types, and improper handling of time alignment can easily introduce leakage; moreover, their inference cost and interpretability are often less transparent than those of gradient-boosted trees.

50%
Unverified

Tabular foundation models are a class of models pretrained on large heterogeneous tabular datasets, with representative work including Kumo and Nori; their typical usage is to organize the samples to be predicted into a single table, and the model directly infers based on the tabular priors obtained from pretraining.

50%
Unverified

Data distribution shift may cause zero-training prediction to underperform models trained specifically for the task, especially when business data has strong domain characteristics (such as medical coding or rare events in financial risk control).

50%
Unverified

When current mainstream tabular foundation models (TFMs) are used directly for survival analysis tasks, none of them can consistently beat the Cox model.

50%
Unverified

A recent study from arXiv proposes using tabular foundation models (TFMs) as surrogate models for Bayesian optimization, requiring no kernel function selection and no additional training for active learning tasks.

50%
Unverified

The paper's conclusion states that the results "motivate developing TFMs further as surrogates for active learning in materials discovery," implying that current TFMs still have room for improvement in uncertainty estimation and acquisition function adaptation in active learning scenarios.

50%
Unverified

For small- to medium-scale datasets, pre-training-based tabular foundation models have the potential to match or even surpass carefully tuned tree models at very low hyperparameter-tuning cost.

50%

Source Articles