Increasing User Trust in Optimisation through Feedback and Interaction.

User trust plays a key role in determining whether autonomous computer applications are relied upon. It will play a key role in the acceptance of emerging AI applications such as optimisation. Two important factors known to affect trust are system transparency, i.e. how well the user understands how the system works, and system performance. However, in the case of optimisation it is difficult for the end-user to understand the underlying algorithms or to judge the quality of the solution. Through two controlled user studies we explore whether the user is better able to calibrate their trust in the system when: (a) they are provided feedback on the system operation in the form of visualisation of intermediate solutions and their quality; (b) they can interactively explore the solution space by modifying the solution returned by the system. We found that showing intermediate solutions can lead to over-trust while interactive exploration leads to more accurately calibrated trust.