Machine learning (ML)-based network intrusion detection is susceptible to attacks that perturb malicious network flows to evade detection. Existing approaches to evaluating the robustness of these models rely on gradient-based optimization that are computationally expensive and restricted to differentiable model architectures. This limits their practicality for continuous, large-scale evaluation. To address this, we develop lightweight adversarial agents trained via reinforcement learning (RL) that decouples the cost of learning an evasion strategy from the cost of executing it. These agents learn offline to perturb malicious NetFlow records to evade surrogate intrusion detection models, encoding the resulting strategy into a reusable policy that requires no gradient computation at deployment. We evaluate our approach on four NetFlow datasets spanning enterprise, cloud, and IoT environments against diverse model architectures, including non-differentiable classifiers that gradient-based methods cannot evaluate directly. Agents achieve up to 58.1