TabAttackBench: A Benchmark for Adversarial Attacks on Tabular Data

Abstract

Adversarial attacks pose a significant threat to machine learning models by inducing incorrect predictions through imperceptible perturbations to input data. While these attacks are well studied in unstructured domains such as images, their behaviour on tabular data remains underexplored due to mixed feature types and complex inter-feature dependencies. This study introduces a comprehensive benchmark that evaluates adversarial attacks on tabular datasets with respect to both effectiveness and imperceptibility. We assess five white-box attack algorithms (FGSM, BIM, PGD, DeepFool, and C&W) across four representative models (LR, MLP, TabTransformer and FT-Transformer) using eleven datasets spanning finance, energy, and healthcare domains. The benchmark employs four quantitative imperceptibility metrics (proximity, sparsity, deviation, and sensitivity) to characterise perturbation realism. The analysis quantifies the trade-off between these two aspects and reveals consistent differences between attack types, with $\ell_{\infty}$-based attacks achieving higher success but lower subtlety, and $\ell_2$-based attacks offering more realistic perturbations. The benchmark findings offer actionable insights for designing more imperceptible adversarial attacks, advancing the understanding of adversarial vulnerability in tabular machine learning.

What this paper studies

An attack that frequently fools a model does not necessarily produce imperceptible perturbations. TabAttackBench examines both sides of that trade-off, comparing attacks across datasets and predictive architectures under a common evaluation framework.

What the benchmark covers

The benchmark connects attack success to four quantitative dimensions of imperceptibility. These dimensions build on the broader framework in our imperceptibility study.

What we found

The evaluated attacks exhibit a trade-off between effectiveness and imperceptibility. In the study, \(\ell_{\infty}\)-based attacks generally achieve higher attack success with less subtle perturbations, while \(\ell_2\)-based attacks offer more realistic perturbations. Dataset characteristics, predictive models and attack settings matter when interpreting these patterns.

This makes the benchmark useful for examining more than an attack-success ranking: it shows what kind of input changes accompany that success.

Using the benchmark

The linked repository provides the implementation, data processing scripts and experimental pipelines. Start with its setup instructions and record the dataset, model, attack configuration and evaluation measures when comparing results.

Scope and limitations

The results concern the five white-box attacks and four predictive architectures evaluated in the paper. They are not a claim to cover every constrained, black-box or model-specific attack. The four quantitative measures also do not replace domain checks for immutability, feasibility and feature dependencies.

Place in my research

TabAttackBench connects the definition of imperceptibility to systematic empirical evaluation. The next question is whether generation methods can better preserve the data distribution, explored in our on-manifold attack paper.

Related research

• Investigating Imperceptibility of Adversarial Attacks on Tabular Data: An Empirical Analysis
• Crafting Imperceptible On-Manifold Adversarial Attacks for Tabular Data

Citation

Cite the published version using 10.1016/j.eswa.2025.130491.

@article{ESWA2026,
  author = {He, Zhipeng and Ouyang, Chun and Wen, Lijie and Liu, Cong and Moreira, Catarina},
  title = {TabAttackBench: A Benchmark for Adversarial Attacks on Tabular Data},
  journal = {Expert Systems with Applications},
  year = {2026},
  volume = {301},
  eid = {130491},
  doi = {10.1016/j.eswa.2025.130491}
}