CADET: Debugging and Fixing Misconfigurations using Counterfactual Reasoning

Modern computing platforms are highly-configurable with thousands of interacting configuration options. However, configuring these systems is challenging and misconfigurations can cause unexpected non-functional faults. This paper proposes CADET (short for Causal Debugging Toolkit) that enables users to identify, explain, and fix the root cause of non-functional faults early and in a principled fashion. CADET builds a causal model by observing the performance of the system under different configurations. Then, it uses casual path extraction followed by counterfactual reasoning over the causal model to (a) identify the root causes of nonfunctional faults, (b) estimate the effects of various configuration options on the performance objective(s), and (c) prescribe candidate repairs to the relevant configuration options to fix the nonfunctional fault. We evaluated CADET on 5 highly-configurable systems by comparing with state-of-the-art configuration optimization and ML-based debugging approaches. The experimental results indicate that CADET can find effective repairs for faults in multiple non-functional properties with (at most) 13% more accuracy, 32% higher gain, and 13× speed-up than other ML-based performance debugging methods. Compared to multi-objective optimization approaches, CADET can find fixes (at most) 8× faster with comparable or better performance gain. Our study of non-functional faults reported in NVIDIA’s forum shows that CADET can find 14% better repairs than the experts’ advice in less than 30 minutes.