PAVE: A Protocol-Validity-aware Framework for Adversarial Robustness Evaluation of Deep-Learning Intrusion Detection in IoT Networks

Network intrusion detection for the Internet of Things increasingly relies on deep-learning classifiers, yet their reported adversarial robustness rests on an untested networking assumption: that a feature-space perturbation corresponds to a packet or flow an attacker can actually transmit on the wire. We propose PAVE (Protocol-Validity-Aware Evaluation), a framework that judges robustness at the realizable network artifact (CAN frames, MQTT packets, IP flows) rather than the normalized feature tensor, coupling a protocol-space validity test, a gradient-masking audit, and seventeen imbalance-aware metrics with corrected paired tests over three IoT datasets, five architectures, and six defenses. PAVE revises standard conclusions: adversarial examples produced by standard unconstrained continuous feature-space attacks (FGSM, PGD, Square) retain only 0%–3% validity in protocol space, so most of the vulnerability these attacks report cannot occur on the wire (enforcing validity shows that unconstrained evaluation overstates the MCC degradation of a Transformer under FGSM by 0.18–0.20: 0.39 vs. 0.56 and 0.30 vs. 0.50 on two benchmarks, i.e. a measurement of evaluation bias, not a defense); its imbalance-aware gradient-masking audit screens for majority-class collapse, a risk we observe for an untuned defense configuration and which principled penalty tuning resolves; and two of three common benchmarks are saturated or leakage-prone (CICIoV2024 reduces to ≈3.6k unique flows from 1.4M). Among low-cost defenses, a feature-space Mahalanobis detector gives strong non-adaptive separation (PR-AUC ≈1.0, FPR@95%TPR ≈0), though it degrades under an adaptive attacker, which PAVE recommends as a complement to, not a replacement for, adversarial training. PAVE provides a realizability-grounded protocol and a more accurate estimate of adversarial risk for IoT intrusion detection.