Large-scale examination of academic publications using statistical models

We describe our experiences in three collaborative visual analytics projects. The projects center on large-scale examination of academic publications using statistical models. Each project involves a multidisciplinary team of social scientists, machine learning researchers, and experts in various academic fields. We highlight design guidelines learned from these experiences. In many cases, effective analysis results only after aligning the visualization with the analysis tasks, the capabilities of the modeling tools, and concepts meaningful to the domain experts. Substantiating findings from the analyses often requires verifying and modifying the underlying statistical models. In the examples below, we demonstrate that alignment allows analysts to verify model outputs by comparing them to domain knowledge and gain trust in the validity of findings derived from the models. Verification often leads to model modifications (changes in assumptions, parameter search, or sensitivity analysis); visualization can expose modeling abstractions, and help model builders better understand and alter the characteristics of the models. Displaying appropriate units of analysis also facilitates communication among collaborators with different backgrounds and expertise.