Stochastic PAC-Bayesian Transformers for Network Intrusion Detection and Natural Language Processing Applications

Transformer architectures dominate contemporary machine learning, yet face critical limitations in security applications: vulnerability to adversarial attacks, lack of calibrated uncertainty estimates, and difficulty distinguishing confident predictions from uncertain cases requiring human review. We introduce stochastic probably approximately correct (PAC) Bayesian transformers that convert deterministic attention into probabilistic variants via variational inference, unifying uncertainty quantification, and adversarial robustness within a single framework. Our approach replaces fixed attention weights with learned variational distributions and propagates uncertainty through Monte Carlo (MC) sampling, creating moving targets that degrade adversarial effectiveness. We derive joint PAC-Bayesian bounds showing that parameter stochasticity improves both calibration and robustness, with complexity scaling as O( √(KL( ρ ||π)/n)) , where KL denotes the Kullback–Leibler divergence between the learned posterior ρ and prior π , and n is the sample size. Across network intrusion detection, toxic content detection, and fake news identification, we achieve 96.8 ± 0.8% accuracy with the expected calibration error (ECE) of 0.043 ± 0.006 , and maintain 88.3 ± 1.5% robust accuracy under multiple attack strategies. Active learning guided by uncertainty reduces labeling requirements by 68% , reaching 95% of full-data performance with only 35% of labels.