SoK: A Survey of Open Source Threat Emulators

Threat emulators are tools or sets of scripts that emulate cyber-attacks or malicious behavior. Specifically, threat emulators can launch single procedure attacks or give one the ability to create multi-step attacks, while the resulting attacks may be known or unknown cyber-attacks. The motivations for using threat emulators are various: cutting costs of penetration testing activities by having smaller red teams, performing automated security audits in organizations, creating baseline tests for security tools in development, supplying penetration testers with another tool in their arsenal, etc. In this paper, we review various open-source threat emulators and perform qualitative and quantitative comparison between them. We focus on tactics and techniques from MITRE ATT&CK matrix, and check if they can be performed and tested with the reviewed emulators. We develop a comprehensive methodology for the evaluation and comparison of threat emulators with respect to general features such as prerequisites, attack manipulation, clean up and more. Finally, we present a discussion on the circumstances in which one threat emulator should be preferred over another. This survey can help security teams, security developers and product deployment teams to examine their network environment or their product with the most suitable threat emulator. Using the provided guidelines, a team can choose the best threat emulator for their needs without checking and trying them all. To the best of our knowledge, no academic comparison was made between threat emulators.