Towards a better understanding of the mechanics of refactoring detection tools

Context: Refactoring is a crucial practice used by many developers, available in popular IDEs, like Eclipse. Moreover, refactoring detection tools, such as RefDiff and RefactoringMiner, help improve the comprehension of refactoring application changes.Objective: In this article, we better understand to what extent refactoring detection tools (RefDiff and RefactoringMiner) identify refactoring operations that developers apply in practice.Methods: We survey with 53 developers of popular Java projects on GitHub. We asked them to identify six refactoring transformations applied to small programs.Results: There is no unanimity in all questions of our survey. Refactoring detection tools do not detect many refactoring operations expected by developers. In 4 out of 6 questions, most developers prefer the Eclipse refactoring mechanics.Conclusion: The results highlight the importance of diving deep into the refactoring mechanics and defining a baseline. Empirical studies focused on mining refactoring operations may be limited by an incomplete or unrepresentative sample of such operations, thus posing a challenge for researchers in this field.