DeepCrime: from Real Faults to Mutation Testing Tool for Deep Learning.

The recent advance of Deep Learning (DL) due to its human-competitive performance in complex and often safety-critical tasks, reveals many gaps in their testing. There exist a number of DL-specific testing approaches, and yet none has presented the possibility of simulating the occurrence of real DL faults for the mutation testing of DL systems. We propose 35 and implement 24 mutation operators that were systematically extracted from the existing studies on real DL faults. Our evaluation shows that the proposed operators produce non-redundant, killable, and non-trivial mutations while being more sensitive to the change in the quality of test data than the existing mutation testing approaches. Video demonstration is available at: https://youtu.be/WOvuPaXH6Jk