Automated Random Testing in Multiple Dispatch Languages

In programming languages that use multiple dispatch, a single function can have multiple implementations, each of which may specialise the function's operation. Which one of these implementations to execute is determined by the data types of all the arguments to the function. Effective testing of functions that use multiple dispatch therefore requires diverse test inputs in terms of the data types of the input's arguments as well as their values. In this paper we describe an approach for generating test inputs where both the values and types are chosen probabilistically. The approach uses reflection to automatically determine how to create inputs with the desired types, and dynamically updates the probability distribution from which types are sampled in order to improve both the test efficiency and efficacy. We evaluate the technique on 247 methods across 9 built-in functions of Julia, a technical computing language that applies multiple dispatch at runtime. In the process, we identify three real faults in these widely-used functions.