An AI Teaching Assistant for Motion Picture Engineering [SP Education]

The rapid rise of LLMs over the last few years has promoted growing experimentation with LLM-driven AI tutors. However the details of implementation, as well as the benefit in a teaching environment, are still in the early days of exploration. This article addresses these issues in the context of implementation of an AI Teaching Assistant (AI-TA) using Retrieval Augmented Generation (RAG) for Trinity College Dublin?s Master?s Motion Picture Engineering (MPE) course. We provide details of our implementation (including the prompt to the LLM, and code [1](#fn-0005)), and highlight how we designed and tuned our RAG pipeline to meet course needs. We describe our survey instruments and report on the impact of the AI-TA through a number of quantitative metrics. The scale of our experiment (43 students, 296 sessions, 1,889 queries over 7 weeks) was sufficient to have confidence in our findings. Unlike previous studies, we experimented with allowing the use of the AI-TA in open-book examinations. Statistical analysis across three exams showed no performance differences regardless of AI-TA access ( p>0.05), demonstrating that thoughtfully designed assessments can maintain academic validity. Student feedback revealed that the AI-TA was beneficial (mean = 4.22/5), while students had mixed feelings about preferring it over human tutoring (mean = 2.78/5).