The first successful detection of gravitational waves by ground-based observatories, such as the Laser Interferometer Gravitational-Wave Observatory (LIGO), marked a breakthrough in our comprehension of the Universe. However, due to the unprecedented sensitivity required to make such observations, gravitational-wave detectors also capture disruptive noise sources called glitches, which can potentially be confused for or mask gravitational-wave signals. To address this problem, a community-science project, Gravity Spy, incorporates human insight and machine learning to classify glitches in LIGO data. The machine-learning classifier, integrated into the project since 2017, has evolved over time to accommodate increasing numbers of glitch classes. Despite its success, limitations have arisen in the ongoing LIGO fourth observing run (O4) due to the architecture's simplicity, which led to poor generalization and inability to handle multi-time window inputs effectively. We propose an advanced classifier for O4 glitches. Using data from previous observing runs, we evaluate different fusion strategies for multi-time window inputs, using label smoothing to counter noisy labels, and enhancing interpretability through attention module-generated weights. Our new O4 classifier shows improved performance, and will enhance glitch classification, aiding in the ongoing exploration of gravitational-wave phenomena.

We present the results of a large-scale participant survey administered to volunteers on the Zooniverse citizen science platform. Nearly three million people have user accounts on zooniverse.org, making it one of the largest platforms for online citizen science. The survey was launched to learn more about the background of volunteers and consisted of four sections: demographics, contribution and engagement, motivations and values, and community perspectives. We received 6,030 valid responses which are reported in the paper. After reporting the results of the survey, we contextualized them through comparisons and contrasts with other citizen science initiatives and the previous Zooniverse survey conducted in 2014.

We explore the bi-directional relationship between human and machine learning in citizen science. Theoretically, the study draws on the zone of proximal development (ZPD) concept, which allows us to describe AI augmentation of human learning, human augmentation of machine learning, and how tasks can be designed to facilitate co-learning. The study takes a design-science approach to explore the design, deployment, and evaluations of the Gravity Spy citizen science project. The findings highlight the challenges and opportunities of co-learning, where both humans and machines contribute to each other’s learning and capabilities. The study takes its point of departure in the literature on co-learning and develops a framework for designing projects where humans and machines mutually enhance each other’s learning. The research contributes to the existing literature by developing a dynamic approach to human-AI augmentation, by emphasizing that the ZPD supports ongoing learning for volunteers and keeps machine learning aligned with evolving data. The approach offers potential benefits for project scalability, participant engagement, and automation considerations while acknowledging the importance of tutorials, community access, and expert involvement in supporting learning.

The Gravity Spy project aims to uncover the origins of glitches, transient bursts of noise that hamper analysis of gravitational-wave data. By using both the work of citizen-science volunteers and machine-learning algorithms, the Gravity Spy project enables reliable classification of glitches. Citizen science and machine learning are intrinsically coupled within the Gravity Spy framework, with machine-learning classifications providing a rapid first-pass classification of the dataset and enabling tiered volunteer training, and volunteer-based classifications verifying the machine classifications, bolstering the machine-learning training set and identifying new morphological classes of glitches. These classifications are now routinely used in studies characterizing the performance of the LIGO gravitational-wave detectors. Providing the volunteers with a training framework that teaches them to classify a wide range of glitches, as well as additional tools to aid their investigations of interesting glitches, empowers them to make discoveries of new classes of glitches. This demonstrates that, when giving suitable support, volunteers can go beyond simple classification tasks to identify new features in data at a level comparable to domain experts. The Gravity Spy project is now providing volunteers with more complicated data that includes auxiliary monitors of the detector to identify the root cause of glitches.

Citizen scientists make valuable contributions to science but need to learn about the data they are working with to be able to perform more advanced tasks. We present a set of design principles for identifying the kinds of background knowledge that are important to support learning at different stages of engagement, drawn from a study of how free/libre open source software developers are guided to create and use documents. Specifically, we suggest that newcomers require help understanding the purpose, form and content of the documents they engage with, while more advanced developers add understanding of information provenance and the boundaries, relevant participants and work processes. We apply those principles in two separate but related studies. In study 1, we analyze the background knowledge presented to volunteers in the Gravity Spy citizen-science project, mapping the resources to the framework and identifying kinds of knowledge that were not initially provided. In study 2, we use the principles proactively to develop design suggestions for Gravity Spy 2.0, which will involve volunteers in analyzing more diverse sources of data. This new project extends the application of the principles by seeking to use them to support understanding of the relationships between documents, not just the documents individually. We conclude by discussing future work, including a planned evaluation of Gravity Spy 2.0 that will provide a further test of the design principles.

We explore how people developing or using a system with a machine-learning (ML) component come to understand the capabilities and challenges of ML. We draw on the social construction of technology (SCOT) tradition to frame our analysis of interviews and discussion board posts involving designers and users of a ML-supported citizen-science crowdsourcing project named Gravity Spy. We extend SCOT by anchoring our investigation in the different uses of the technology. We find that the type of understandings achieved by groups having less interaction with the technology is shaped more by outside influences and less by the specifics of the system and its role in the project. This initial understanding of how different participants understand and engage with ML points to challenges that need to be overcome to help users of a system deal with the opaque position that ML often holds in a work system.

In this article, we present a study of scientific discovery through trace data. Using data produced from (1) scientists’ interactions on computer systems and (2) meeting transcripts generated from weekly project meetings, we analyzed the interactions and conversations of scientists surrounding a scientific discovery to determine whether trace data could provide markers for the discovery. The results describe the process of interacting with computer tools and disseminating results and highlight such markers. This study shows trace data, and advanced computational techniques are useful mechanisms for identifying work trajectories and, in some cases, provide context to work behaviors.

Peer production projects involve people in many tasks, from editing articles to analyzing datasets. To facilitate mastery of these practices, projects offer a number of learning resources, ranging from project-defined FAQsto individually-oriented search tools and communal discussion boards. However, it is not clear which project resources best support participant learning, overall and at different stages of engagement. We draw onSørensen's framework of forms of presence to distinguish three types of engagement with learning resources:authoritative, agent-centered and communal. We assigned resources from the Gravity Spy citizen-science into these three categories and analyzed trace data recording interactions with resources using a mixed-effects logistic regression with volunteer performance as an outcome variable. The findings suggest that engagement with authoritative resources (e.g., those constructed by project organizers) facilitates performance initially. However, as tasks become more difficult, volunteers seek and benefit from engagement with their own agent-centered resources and community-generated resources. These findings suggest a broader scope for the design of learning resources for peer production

The voluntary nature of participation in many open collaboration platforms means platform managers and system designers need to develop strategies to attract new participants and retain existing ones. Implementing features that positively impact users’ motivational states will help address challenges surrounding user recruitment and attrition. This paper reports on the design, implementation, and evaluation of one such feature – novelty cues. Prior studies have shown that novel stimuli attract and increase attention towards objects. Citizen science projects regularly ask volunteers to classify data that no other human has seen previously. An experiment was designed where volunteers were shown novelty cues. The goal was to evaluate the saliency of novelty in motivating participation among citizen science volunteers. The results showed that under most circumstances, novelty cues are effective mechanisms to increase user motivation; however, its effectiveness may be mediated by other factors such as existing system design and individual preferences for novelty.

Researchers studying user behaviors in online communities often conduct analyses of user interaction data recorded in system logs e.g., an edit in Wikipedia. Such analysis relies on collating interactions by a unique identifier such as a user ID. However, if users can contribute without being logged-in (i.e., anonymously) analysis of interaction data omit part of a user's experience. Problematically, anonymous traces are unlikely to be randomly distributed, so their omission can change statistical conclusions, with implications for both research and practice. To understand the impacts on conclusions of leaving out anonymous traces, we conducted an analysis of system logs from two online citizen science projects. Attributing anonymous traces with user IDs, we found that (1) many users contribute anonymously, though with varied patterns; and (2) attributing anonymous traces diminishes empirical evidence used to support theory and change the results of system algorithms. These results suggest anonymous traces have implications for research on user behaviors and the practices associated with using such data to tailor user experiences in online communities.

Privacy behaviors of individuals are often inconsistent with their stated attitudes, a phenomenon known as the "privacy paradox." These inconsistencies may lead to troublesome or regrettable experiences. To help people address these privacy inconsistencies, we propose a personalized privacy notification approach that juxtaposes users' general privacy attitudes towards specific technologies and the potential privacy riskiness of particular instances of such technology, right when users make decisions about whether and/or how to use the technology under consideration. Highlighting the privacy inconsistencies to users was designed to nudge them in making decisions in a way that aligns with their privacy attitudes. To illustrate this approach, we chose the domain of mobile apps and designed a privacy discrepancy interface that highlights this discrepancy between users' general privacy attitudes towards mobile apps and the potential privacy riskiness of a particular app, nudging them to make app installation and/or permission granting decisions reflecting their privacy attitudes. To evaluate this interface, we conducted an online experiment simulating the process of installing Android apps. We compared the privacy discrepancy approach with several existing privacy notification approaches. Our results suggest that the behaviors of participants who used the privacy discrepancy interface better reflected their privacy attitudes than the other approaches.

Increasingly, citizen scientists do work beyond the primary goal of the project (i.e., advanced work) such as writing articles. These activities often take place in discussion boards and have a set of linguistic norms for contributing. For newcomers, learning this language presents a challenge since there are no formal opportunities for them to learn the language and volunteers who join later need to learn more than volunteers who join earlier in a project life-cycle. In this poster, we examine how newcomers language use shifts over the course of two citizen science projects. We find that, although, newcomers joining later might face obstacles, newcomer language associated with advanced work increase over the project's life-cycle. The analysis can help the science team assess whether newcomers on the talk page have either adopted advanced terminologies or they need to have a more formal resource such as tutorial or blog posts.

When people interact via information systems, the data is captured by the systems as a side effect of the interaction. These data are increasingly interesting and available for research. In a sense, these systems become a new kind of research apparatus, and like all advances in instrumentation, open up new areas of study with the potential for discovery. While at first glance, such “big data” analysis seems to be most suitable for a positivist quantitative research approach. However, a closer inspection reveals that interpretive research strategies may better support the challenges associated with digital trace data. By merging insights from hermeneutics and sociomateriality, we argue that trace data analysis entails the building of a research apparatus. Hermeneutic principles play a key role in the application of this apparatus and allow researchers to make sense of the often partial traces left by online participants. Drawing on longitudinal trace data from a study of citizen science practices the paper illustrates the value of merging insights from hermeneutics with sociomaterial insights. The approach allows researchers to account for not only the material dynamics of digital trace data but also the temporal dimension of online practices.

In this paper, we describe the results of an online field experiment examining the impacts of messaging about task novelty on the volume of volunteers' contributions to an online citizen science project. Encouraging volunteers to provide a little more content as they work is an attractive strategy to increase the community's output. Prior research found that an important motivation for participation in online citizen science is the wonder of being the first person to observe a particular image. To appeal to this motivation, a pop-up message was added to an online citizen science project that alerted volunteers when they were the first to annotate a particular image. Our analysis reveals that new volunteers who saw these messages increased the volume of annotations they contributed. The results of our study suggest an additional strategy to increase the amount of work volunteers contribute to online communities and citizen science projects specifically.