Managing the Chaos: Approaches to Navigating Discussion Forums for Instructional Staff

As enrollment increases, teaching assistants (TAs) need help prioritizing responding to students' posts in on-line forums. In a graduate-level Artificial Intelligence forum, three instructors rank the urgency of posts which is compared to the ratings of 13 course TAs; correlation with the instructors' scores have an r=0.55, with a TA inter-rater reliability of 35%. However, when TAs used a codebook containing seven dimensions created by the instructors to define urgency levels, correlation increased to r=0.73 and reliability to 53%. The instructor rankings are also compared to cognitive presence ratings from the Community of Inquiry framework. Cognitive presence correlates with urgency with r=-0.68. These results suggest that recommendation agents that prioritize posts based on cognitive presence or the urgency codebook may be beneficial for TAs.