XEMU: an Efficient QEMU Based Binary Mutation Testing Framework for Embedded Software

This paper presents the XEMU framework for mutation based testing of embedded software binaries. We apply an extension of the QEMU software emulator, which injects mutations at run-time by dynamic code translation without affecting the binary software under test. The injection is based on a mutation table, which is generated by control flow graph (CFG) analysis of the disassembled code prior to its execution without presuming access to source code. We introduce our approach by the example of the ARM instruction set architecture for which a mutation taxonomy is presented. In addition to extending the testing scope to target specific low level faults, XEMU addresses the reduction of the mutants creation, execution, and detection overheads. Moreover, we reduce testing efforts by applying binary CFG analysis and constraint-based test generation for improved test quality. The experimental results of a car motor management software show significant improvements over conventional source code based approaches while providing 100% accuracy in terms of the computed test quality metrics.