1/ When Do Neural Nets Outperform Boosted Trees on Tabular Data?
2/ there is still an active discussion on whether or not NNs generally outperform gradient-boosted decision trees (GBDTs) on tabular data, with several recent works arguing either that GBDTs consistently outperform NNs on tabular data, or vice versa.
3/ "In this work, we take a step back and question the importance of this debate. To this end, we conduct the largest tabular data analysis to date, comparing 19 algorithms across 176 datasets, and we find that the ‘NN vs. GBDT’ debate is overemphasized"
for a surprisingly high number of datasets, either the performance difference between GBDTs and NNs is negligible, or light hyperparameter tuning on a GBDT is more important than choosing between NNs and GBDTs. https://proceedings.neurips.cc/paper_files/paper/2023/hash/f06d5ebd4ff40b40dd97e30cee632123-Abstract-Datasets_and_Benchmarks.html
#machinelearning
@tangming2005 Absolutely agree with this. I've recently scrapped a (fairly complex and difficult to interpret) NN pipeline for transcriptomics analysis for a much simpler XGBoost pipeline and performance is the same or higher in all cases.
Also, interpretability is much higher.
