Proposal of a new fidelity measure between computed image quality and observers quality scores accounting for scores variability

Assessment of the visual quality of colour images is usually a difficult process, validated through hard-to-carry-out psychophysical experiments, used to record observer quality scores. Visual image quality metrics aim to maximise the agreement between computed indexes and observer scores, or opinions. Therefore, in this area, it is of critical importance to have appropriate measures of this agreement (i.e. performance) between the computed image quality metric values and observer’s quality scores, both for the development, as well as for the use of image quality metrics. Among the measures of agreement, the most used one nowadays is the well-known Pearson correlation coefficient, while Spearman rank correlation coefficient is also commonly used. The aim of this paper is two-fold. First, to introduce the Standardised Residual Sum of Squares (STRESS) as an alternative metric for the agreement between computed image quality and observers quality scores and analyse its properties and advantages in front of Pearson, Spearman and Kendall correlation coefficients; Second, to introduce a new version of STRESS (called USTRESS) that takes observers’ scores variability into account. The results on synthetic and real datasets support that STRESS has a series of benefits in front of the classical approaches and that the inclusion of uncertainty in STRESS has an important effect on the results, quantified by statistical significance tests. A free to download MATLAB code version of USTRESS is available at https://viplab.webs.upv.es/resources/