Poster: Transformer-Based Detection of Code Obfuscation Techniques Using Opcode Sequences | AMiner
Poster: Transformer-Based Detection of Code Obfuscation Techniques Using Opcode Sequences
Inwoo Jo,Wanju Kim,Eun-Sun Cho
2026 IEEE International Conference on Software Testing, Verification and Validation (ICST)(2026)
Dept. of Computer Sci. & Eng.
被引用0|浏览0
摘要
Code obfuscation is widely used by malware to evade analysis and detection. Modern malware often applies multiple obfuscation techniques simultaneously, making it necessary to identify the set of techniques used in a binary. This task can be naturally formulated as a multi-label classification problem. Existing approaches commonly rely on opcode frequency statistics or handcrafted features, but they ignore the sequential context of instructions and therefore have limited ability to capture structural patterns introduced by obfuscation. In this paper, we propose a Transformer-based method for detecting code obfuscation techniques from opcode sequences. We construct a dataset by applying representative O-LLVM obfuscation techniques—Bogus Control Flow (BCF), Control Flow Flattening (FLA), and Instruction Substitution (SUB)—to programs from the GNU Coreutils suite. Opcode sequences are extracted through disassembly, segmented with a sliding-window strategy, and processed by a DistilBERT encoder in a multi-label classification framework. Experimental results show that the proposed approach learns characteristic patterns of different obfuscation techniques and achieves strong performance even in multi-obfuscation settings. These findings suggest that contextual modeling of opcode sequences with Transformer architectures can effectively complement traditional statistical approaches to obfuscation detection.