Knowledge Of Problem Solving (KOPS) scale: Design and evaluation of a scalable competency measure for a common practice element in task-shared youth mental health interventions

Objectives: Quality assurance methods for psychological interventions conventionally rely on expert-led assessments that are unfeasible for task-sharing. The current study was conducted in India to design and test a scalable therapeutic competency measure for use with non-specialists before and after training in an evidence-based, adolescent-focused problem-solving intervention. Methods: The developmental phase established a set of key competencies associated with problem-solving therapies, followed by item generation for a corresponding knowledge-based competency measure (the Knowledge of Problem Solving [KOPS] scale). Items were refined in a reference sample (N=31) that included experienced counsellors and individuals without mental health care experience. The resulting 17-item measure was structured around a series of session vignettes for a hypothetical case, with each vignette followed by 3-4 multiple questions. The validation phase evaluated two versions of the measure in a sample of 277 trainees from universities and non-governmental organisations before and after training. Results: Rasch analysis revealed similar results for the two forms. The difficulty level of the test items was well matched to the ability level of participants (i.e., most items being of moderate difficulty and few items being easy or difficult). Most items had a probability below 0.5 suggesting that a respondent of average ability would choose an incorrect response more often than a correct one. Only one item showed a negative discrimination index and was removed from the final 16-item versions. Conclusions: The KOPS measure provides a scalable method to assess key competencies required to deliver transdiagnostic problem-solving therapies. Similar formats could be developed and deployed to support scaling up of other common elements involved in evidence-based psychological interventions globally.