TabPFN vs XGBoost

Two different kinds of model

XGBoost, described by Chen and Guestrin in 2016, builds trees sequentially, each correcting the errors of the ensemble so far, with engineering for sparse data, approximate split finding and out-of-core computation. The fitted model is specific to one dataset. Its documentation states that missing values are handled by learning a default branch direction, and that the library scales through distributed frameworks when data exceeds memory.

TabPFN inverts the workflow. The expensive learning happens once, at pretraining, on a large collection of synthetic tabular tasks. At use time the labelled rows are the prompt. The 2022 paper claimed classification on small datasets "in less than a second" with no hyperparameter tuning. The 2025 Nature paper reported that, on datasets up to 10,000 samples and 500 features, TabPFN in 2.8 seconds outperformed an ensemble of the strongest baselines tuned for four hours.

What independent benchmarks say

For years the consensus ran the other way. Grinsztajn, Oyallon and Varoquaux (2022) found tree-based models remained state of the art on medium-sized tabular data of about 10,000 samples, and attributed it to inductive biases neural networks lacked. TabArena, a continuously maintained benchmark of 51 curated datasets (Erickson et al., NeurIPS 2025), updated the picture: gradient-boosted trees "are still strong contenders", deep learning methods caught up under larger time budgets with ensembling, and "foundation models excel on smaller datasets". At launch TabPFNv2 could be run on only 33 of the 51 datasets because of its size limits, and within that subset it led by a wide margin.

Two notes on independence. TabArena's authors include Frank Hutter, a TabPFN co-author, although the benchmark is a multi-institution effort with public code. And the current live leaderboard could not be read for this page, so standings after the launch paper are taken from Prior Labs' own reports.

Vendor-reported progress since 2025

Prior Labs' TabPFN-2.5 report (November 2025) claims the top TabArena position, a 100% win rate against default XGBoost on small-to-medium classification datasets and 87% on datasets up to 100,000 rows. The comparison is with default, not tuned, XGBoost. The TabPFN-3.5 technical report (September 2026) claims first place on TabArena and gains of about 150 Elo on a broader vendor benchmark, with variants for text-rich columns, maximum accuracy and speed. That report gives no direct numerical comparison with tuned XGBoost, LightGBM or CatBoost, and its claims have not, as far as could be found, been independently reproduced.

Scale, cost and licensing

TabPFN's stated envelope has grown from 1,000 training points in 2022 to 1,000,000 rows in the 3.5 README. The published evidence at the top of that range is thinner than at the bottom. Because the training table is processed at prediction time, the Nature paper notes that memory scales with dataset size and that the model is not optimized for real-time inference. Prior Labs addresses this with a distillation engine that converts a fitted TabPFN into a compact MLP or tree ensemble, and with a faster 3.5 variant reported as up to six times quicker than the base model.

Licensing is a practical difference. XGBoost is Apache 2.0 throughout. The TabPFN code is Apache 2.0, but the repository states that weights for versions 2.5 through 3.5 are released under non-commercial licences, with commercial use available through an enterprise licence or the hosted API. Only the older v2 weights carry a permissive licence with an attribution requirement.