Assessment of Visual Aesthetics Through Human Judgments: a Systematic Mapping

Visual aesthetics is an essential factor of the perceived usability, credibility, and general quality of graphical user interfaces (GUIs). Its assessment as part of the software process is critical but highly subjective, making it difficult to obtain reliable and valid results. This article presents a systematic mapping analyzing measures and data collection instruments to understand how humans can assess GUI visual aesthetics with reliability and validity. We encountered 148 studies assessing visual aesthetics through human judgments, either as an unidimensional or a multidimensional construct. Most studies that assess visual aesthetics as a composition of multiple dimensions apply the classical/expressive or the VisAWI questionnaire. Yet, we observed large variability regarding research design, including the number of assessed GUIs, ratings, and raters. Although most studies report demographic data, very few analyze their influence on the results. Less than half of the studies evaluate reliability or validity. We expect the results to contribute to subjective assessments of GUI visual aesthetics, help practitioners select and use appropriate methods, and point out research opportunities in this area.