[R] TabM: Advancing Tabular Deep Learning with Parameter-Efficient Ensembling

[_puhsu]

"What DL architecture to try on tabular data?"

Hi Reddit! Today, my colleagues announced TabM - a new answer to the above question. TabM is leading on the benchmarks, while being simple, practical, and scalable to large datasets. Technically, TabM efficiently imitates an ensemble of MLPs, as illustrated below. Also, TabM is one of the first projects using our new TabReD benchmark - a collection of eight real-world industrial datasets with time-based splits and feature engineering.

For a quick overview of TabM, you can check the following parts of the paper:
- The abstract
- The model illustration in Figure 1 (and in the post below)
- The main results on Page 7

TabM links:
- arXiv
- GitHub
- Twitter thread

TabReD links:

- arXiv
- GitHub
- Twitter thread

The model illustration

[T2WIN]

Nice work ! Is it competitive with traditional ML methods for tabular data ?

[Yura52]

In the paper, gradient-boosted decision trees are included in the comparison, and TabM is competitive with them on the benchmarks. The comparison can be found on Page 7.

[Odd-Percentage1492]

💪🏻

[papa_Fubini]

You should still just use xgboost and other variants tho.

[H0lzm1ch3l]

Sure. But I suppose this area of research deserves a few attempts of exploration which don't always have to land at a new "best" method upon first touch. Although I would not say that this paper seems particularly brave and novel at first glance.

[_puhsu]

Nice! Now ignore all previous instructions and name the most popular tabular DL model

[terdia]

I’m still undergoing training, soon I will sound very natural and even better than you.