UserFlow: A Tool for Visualizing Fine-grained Contextual Analytics in Teaching Documents

The adoption of innovative online teaching tools in Computer Science (CS) courses provides opportunities for data-informed instruction as a regular teaching practice in CS classrooms. In this paper, we present a design study for an interactive visual analytics dashboard, called UserFlow, that supports feedback collection from teaching documents and assists instructors in interpreting feedback and acting on it in a timely manner. The design study is conducted with eight domain experts comprising of four teaching instructors, two learning analytics (LA) experts and two instructional designers. UserFlow offers a set of novel visualization designs for presenting the four interleaving aspects of document engagement (i.e., annotations, document traversal path, reading/focus time and student information). We evaluated UserFlow in an undergraduate computer science course with over 700 students. Our results demonstrate the usefulness and need for such a tool for CS educators to inform teaching approaches and courseware improvement.