A Multi-Trait Multi-Method Examination of Psychometric Instrument Performance in Autism Spectrum Disorder

Both the Autism Diagnostic Observation Scale (ADOS), a performance-based measure, and the Social Responsiveness Scale (SRS), a report-based measure, were developed to assess the core symptoms of Autism Spectrum Disorder (ASD), yet they have been shown in multiple datasets not to correlate as would be expected if each were measuring the same construct. Moreover, anecdotal evidence has suggested that report-based measures may have strong across-trait/within-individual covariance that detracts from between-trait measurement precision. We used a multi-trait, multi-method approach to examine method-associated bias within an ASD sample (n = 83). We examined performance/rater instrument pairs for attention, inhibition, working memory and motor coordination. Rater-based scores showed an overall greater methodology bias (47% of variance in score explained by method), while performance-based scores showed a weaker methodology bias (28%). The degree of inter-individual variance explained by method alone verifies an anecdotal concern associated with the use of rater measures in ASD.