Investigating the Impact of Group Size on Non-Programming Exercises in CS Education Courses

Computer science (CS) courses are taught with increasing emphasis on group work and with non-programming exercises facilitating peer-based learning, computational thinking, and problem solving. However, relatively little work has been done to investigate the interaction of group work and non-programming exercises because collaborative, non-programming work is usually open-ended and requires analysis of unstructured, natural language responses. In this paper, we consider collaborative, non-programming work consisting of online wiki text from 236 groups in nine different CS1 and higher-level courses at a large Midwestern university. Our investigation uses analysis tools with natural language processing (NLP) and statistical analysis components. First, NLP uses IBM Watson Personality Insights to automatically convert students' collaborative wiki text into a Big Five model. This model is useful as a quality metric on group work since Big Five factors such as Openness and Conscientiousness are strongly related to both academic performance and learning. Then, statistical analysis generates regression models on group size and each Big Five trait that make up the factors. Our results show that increasing group size has a significant impact on collaborative, non-programming work in CS1 courses, but not for such work in higher-level courses. Furthermore, increasing group size can have either a positive or negative impact on the Big Five traits. These findings imply the feasibility of using such tools to automatically assess the quality of non-programming group exercises and offer evidence for effective group sizes.