# Tabular Foundation Models

> Tabular foundation models are neural networks pretrained across many tables that predict on a new table in one forward pass, without per-task training.

Source: https://metavert.io/tabular-foundation-models  
Published: 2026-10-07  
Updated: 2026-10-07

**Tabular foundation models** are neural networks pretrained once, across millions of tables, so that they can make predictions on a new spreadsheet-style dataset without being trained on it: the labelled rows are passed in as context and the model outputs predictions for the unlabelled rows in a single forward pass. They are the first serious challenge in two decades to gradient-boosted decision trees such as [XGBoost](https://metavert.io/xgboost), LightGBM and CatBoost on structured data, and as of October 2026 they lead the main public benchmark for small and medium tables. They have not made boosted trees obsolete: the strongest models carry licence restrictions, need a GPU, and move the computational cost from training to prediction time.

### How They Work: Prior-Data Fitted Networks

The dominant recipe is the prior-data fitted network (PFN). Instead of collecting real tables, the developers define a prior, a generator of synthetic prediction problems, and train a [transformer](https://metavert.io/transformer) on a very large number of them to predict held-out labels given the rest of the table. The original TabPFN paper (Hollmann, Müller, Eggensperger and Hutter, July 2022) describes the result as a model that takes training and test samples as one set-valued input and returns predictions in a single forward pass. This is in-context learning, the same mechanism by which [large language models](https://metavert.io/large-language-models) use examples in a prompt, applied to rows and columns. There is no gradient descent on the user's data and, by default, no hyperparameter search.

The Nature paper on TabPFN v2 (Hollmann et al., January 2025) reports pretraining on roughly 130 million synthetic datasets generated from structural causal models, with an attention scheme in which each cell attends first along its row and then along its column. Not every model in the class is purely synthetic: TabDPT (Ma et al., NeurIPS 2025) pretrains on real tables and argues that real data carries signal synthetic priors miss.

### The Main Models

| Model | Origin | Notable property (as reported by its authors) |
| --- | --- | --- |
| [TabPFN](https://metavert.io/tabpfn) family | University of Freiburg, then Prior Labs | v2 (Nature, 2025) targeted up to 10,000 rows; TabPFN-3.5 (September 2026) accepts up to 1,000,000 rows and 20,000 features |
| TabICL / TabICLv2 | Qu, Holzmüller, Varoquaux, Le Morvan | Column-then-row attention compresses each row to a fixed embedding before in-context learning; v2 (ICML 2026) releases inference and pretraining code |
| Mitra-v2 | Amazon researchers | 77-million-parameter model trained only on synthetic data; weights and code under Apache-2.0 |
| TabDPT | Ma et al. | Retrieval-based in-context learning pretrained on real tables; open weights and training code |

### Benchmark Standing

The reference benchmark is TabArena (Erickson et al., NeurIPS 2025 Datasets and Benchmarks), a continuously maintained leaderboard built from 51 curated datasets with between 500 and 250,000 training rows. Its paper concluded that "tabular foundation models dominate for small data," that gradient-boosted trees "are still strong contenders," and that tuned, ensembled deep-learning models had caught up with them. It also recorded an awkward detail: the foundation models of the time could not run everywhere, so TabPFN v2 was scored on 33 of the datasets and TabICL on 36.

Since then the leaderboard has changed hands several times, and almost every claim comes from the model's own authors. TabICLv2 (February 2026) reported surpassing the then-leading RealTabPFN-2.5 without tuning. Prior Labs' TabPFN-3 report (May 2026) said a single forward pass beat all other models, including tuned and ensembled baselines. Its TabPFN-3.5 release (September 2026) states that the model "ranks first place on both TabArena and BeyondArena," the latter a 142-dataset suite the company assembled around grouped, temporal, high-cardinality and text-rich data. These are vendor-reported results; the leaderboard is public, but readers should check the live standings rather than rely on any release note.

### Limits

Three constraints matter in practice. The first is **size**. Row and feature ceilings have risen quickly, from 1,000 rows in 2022 to a million in 2026, but in-context learning still holds the whole training set in memory at prediction time; the Nature paper notes that memory grows with dataset size. The second is **latency and hardware**. A boosted tree is slow to tune and fast to score; a tabular foundation model reverses that. The TabPFN repository recommends a GPU and limits CPU use to about 5,000 samples for its current models. Distilling the model into a small network or tree ensemble is the usual answer for real-time serving. The third is **licensing**. The code is often open while the best weights are not: TabPFN-2.5 and later are released under non-commercial licences, whereas TabPFN v2, TabICLv2 and Mitra-v2 are more permissive. Being [open-weight](https://metavert.io/open-weight-models) is not the same as being free to deploy.

### When Gradient-Boosted Trees Still Win

Boosted trees remain the sensible default when the table has many millions of rows, when predictions must be served on CPUs within milliseconds, when a permissive licence is mandatory, or when a tuned and monitored model already exists and the expected gain is small. They also come with mature tooling for monotonic constraints, explanations and incremental retraining. Tabular foundation models are strongest where data is scarce, where tuning time is the bottleneck, or where many small models must be built quickly, which is why they are being tested for tasks such as [churn prediction](https://metavert.io/churn-prediction) and [lifetime value](https://metavert.io/customer-lifetime-value) estimation. The honest procedure is a head-to-head comparison on one's own data with time-ordered splits; public leaderboards measure average rank across datasets, not performance on a particular table. The trade-offs are set out in [TabPFN vs XGBoost](https://metavert.io/compare/tabpfn-vs-xgboost).

## Related Topics

- [TabPFN](https://metavert.io/tabpfn) — the model family that established the category
- [XGBoost](https://metavert.io/xgboost) — the gradient-boosted baseline these models are measured against
- [TabPFN vs XGBoost](https://metavert.io/compare/tabpfn-vs-xgboost) — side-by-side trade-offs
- [Foundation Models](https://metavert.io/foundation-models) — the broader pretrain-once, reuse-everywhere pattern
- [Behavioral Foundation Models](https://metavert.io/behavioral-foundation-models) — the equivalent idea for event sequences rather than tables
- [Synthetic Data](https://metavert.io/synthetic-data) — what most of these models are pretrained on
- [Churn Prediction](https://metavert.io/churn-prediction) — a typical small-to-medium tabular task
- [Predictive Analytics](https://metavert.io/predictive-analytics) — the application area these models serve
- [AI Benchmarks](https://metavert.io/ai-benchmarks) — how leaderboards such as TabArena should be read

## Further Reading

- [TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Second](https://arxiv.org/abs/2207.01848) — Hollmann et al., arXiv, July 2022
- [Accurate predictions on small data with a tabular foundation model](https://pmc.ncbi.nlm.nih.gov/articles/PMC11711098/) — Hollmann et al., Nature, January 2025
- [TabArena: A Living Benchmark for Machine Learning on Tabular Data](https://arxiv.org/abs/2506.16791) — Erickson et al., NeurIPS 2025 (arXiv v4, November 2025)
- [TabICL: A Tabular Foundation Model for In-Context Learning on Large Data](https://arxiv.org/abs/2502.05564) — Qu et al., ICML 2025
- [TabICLv2: A better, faster, scalable, and open tabular foundation model](https://arxiv.org/abs/2602.11139) — Qu et al., arXiv, February 2026
- [TabDPT: Scaling Tabular Foundation Models on Real Data](https://arxiv.org/abs/2410.18164) — Ma et al., NeurIPS 2025
- [Mitra-v2 Technical Report](https://arxiv.org/abs/2609.04540) — Tao et al., arXiv, September 2026
- [TabPFN-3: Technical Report](https://arxiv.org/abs/2605.13986) — Grinsztajn et al., arXiv, May 2026
- [TabPFN-3.5 release notes and technical report](https://priorlabs.ai/technical-reports/tabpfn-3-5) — Prior Labs, September 2026
- [TabPFN repository (limits, hardware guidance, licences)](https://github.com/PriorLabs/TabPFN) — Prior Labs / GitHub, accessed October 2026