Using Generative AI to Provide Feedback to Adult Tutors in Training and Assess Real-life Performance

Tutoring remains among the most impactful academic interventions known to improve student achievement. Despite its success, there are few available opportunities for training and professional development for adult tutors. Furthermore, the task of assessing tutor performance during actual tutoring sessions is challenging and time-consuming for human evaluators—enter artificial intelligence (AI). In this work, we harness generative AI, specifi-cally large language models (LLMs) like GPT-4, to introduce an innovative approach for delivering tutors real-time, explanatory feedback while tutors engage in online scenario-based lessons. Hosted within the Personalized Learning Squared (PLUS) tutoring platform, these lessons offer tutors op-portunities to practice responding to common tutoring scenarios. For exam-ple, in the Giving Effective Praise lesson, tutors practice responding to students by providing praise and then receiving immediate and templated feedback generated by LLMs. Beyond demonstrating tutor training contain-ing AI-generated feedback, we report past work using LLMs to assess tutors providing praise, revealing moderate performance, and explaining several prompting methods. While using generative AI shows promise as a low-cost and efficient method for giving adult tutors feedback on their perfor-mance in training and actual tutoring, it is not without limitations. Practical and ethical considerations are discussed. Human tutoring remains a robust and irreplaceable influence on student learning, with generative AI serving as a valuable tool to complement tutors and enhance the impact of tutoring on student learning.