Persona Prototypes for Improving the Qualitative Evaluation of Recommendation Systems

The majority of existing research in the field of recommendation systems is aimed at optimizing accuracy metrics for given datasets, which leads to an algorithm-driven design of resulting solutions. Given a lack of understanding of the dataset characteristics and insufficient diversity of represented individuals, such approaches lead to amplifying the hidden data biases and existing disparities. In this research, we address this problem by proposing a Persona Prototyping approach that selects a set of the most representative user individuals to help in understanding the complex distribution of user interests and performing a proper qualitative evaluation of recommendation algorithms. A hierarchical density-based clustering technique is applied to distinguish diverse user groups and select their prototypes. Each of the selected representatives is presented in an easily understandable form of a textual user story describing the prototype behaviors, inspired by the concept of persona from the interaction design. We evaluated the diversity and representativeness of selected individuals and the results show that the proposed method is capable of identifying diverse interest archetypes and can be used to improve the qualitative analysis of recommendations and to test how well they respond to the diversity of user needs.