Asking crowds of stakeholders to generate and prioritize ideas has become commonplace, from citizen assemblies to corporate open-innovation challenges. Although the reliability of the results of such consultations is taken for granted, their consistency across independent groups has received little empirical attention, largely because such processes are typically one-off events that cannot easily be repeated. In this exploratory study, we introduce the concept of parallel-crowds reliability, the consistency of outcomes when the same collective intelligence process is applied to the same problem by different but demographically equivalent crowds, and we provide an initial empirical test of it. Two separate but equivalent groups of MBA students in the same course were asked to independently propose and prioritize ideas to improve their MBA program, with the same decision-maker curating and selecting winning ideas in both groups. We assess reliability at three successive stages using semantic similarity measures based on two independent embedding models. We find that the two groups independently proposed semantically similar ideas, with convergence concentrated among valuable ideas and divergence among poor ones. They also prioritized similar ideas similarly, though this correspondence was concentrated among the ideas they judged most valuable, enabling the decision maker to select similar sets of winning ideas. Given the single-domain design, we frame these results as an initial evidence base for future investigations.
Background: Collective self-organising behaviour is ubiquitous in nature, whereby complex patterns emerge from the local interactions between individuals. Yet in humans, most group behaviour is attributed to explicit central control or pre-planning, rather than to the synergistic decentralised interplay between individuals. Purpose: This paper introduces Synch.Live , a novel experimental paradigm for quantitatively studying collective motion in humans, framed as a game with an unspecified task and a group feedback mechanism that can be solved through cooperation by 10 participants walking together. We also present the results of a pilot study demonstrating its use. Study sample : 10 partitipants took part in each trial, forming 16 independent groups. Data collection and/or Analysis: We collected psychometrics, behavioural and trahectory data in order to explore the participants’ state of mind and the strategies they developed collectively to solve the group challenge. Results: We found that more than half of the groups participating in the study self-organised to achieve collective flocking motion, and winning players showed higher connectedness to others compared to those who failed. Furthermore, individuals with an awareness of working strategies were more likely to be part of winning groups, suggesting the importance of individual contributions to the collective task. Conclusion: This work demonstrates that solving an unspecified group challenge in response to group feedback is possible, and moreover, that flock-like collective movement has the potential to yield social benefits and well-being, suggesting new directions for exploring social aspects of consciousness and cognition.
Background A growing body of research has identified behavioral markers of collective intelligence to diagnose the quality of group collaboration in real time. Prior work suggests that real-time collaborative process metrics may be used not only to measure and predict the collective intelligence of a group, but also to improve team performance through targeted interventions. Purpose This research examines whether providing teams with real-time feedback on their collaborative process metrics improves performance and collective intelligence in a dynamic team task. Research Design We conducted two experiments in which teams collaborated on an online search and rescue task. Experimental conditions varied whether teams were exposed to real-time displays of their collaborative process metrics. Study Sample Participants were organized into teams and completed the task in an online experimental environment. Data Collection and/or Analysis We collected behavioral trace data capturing team interactions and computed real-time collaborative process metrics. Team performance and collective intelligence were analyzed across experimental conditions to assess the effects of feedback exposure. Results Contrary to expectations, real-time feedback did not uniformly improve collaboration quality or performance. In several instances, teams over-corrected their behavior in response to feedback on a single dimension, which negatively impacted other dimensions of collaboration and reduced task performance. Conclusion These findings suggest that real-time digital nudges aimed at improving collaboration can produce unintended consequences. Caution is warranted when deploying process-level, real-time interventions, as optimizing one collaborative dimension may disrupt others and undermine collective intelligence.
In social learning models, truth-seeking agents learn both individually from direct evidence and socially by pooling beliefs with others. That learning can be undermined by two types of unreliable agents: zealots, who do not learn and promote the same fixed opinion and free riders, who lack access to evidence yet still influence others. In this paper, we explore how learning rules that incorporate memory can mitigate the effects of unreliable agents. To do so, we construct an agent-based model of social learning in which agents apply a probabilistic bounded confidence (BC) model that evaluates the similarity between themselves and others based on samples of recent beliefs rather than current beliefs only. When compared to a memoryless BC benchmark, BC with memory proves significantly less sensitive to the choice of similarity threshold governing agent interactions, to the extent that a fixed threshold is effective for avoiding all types of zealots. It is also less susceptible to high levels of distrust in evidence. The BC with memory model is then extended to social learning about multiple hypotheses, and we show that the robustness results generalise to the case in which beliefs are multi-dimensional probability distributions.
Scientists must choose which among many experiments to perform. We study the epistemic success of experimental choice strategies proposed by philosophers of science or executed by scientists themselves. We develop a multi-agent model of the scientific process that jointly formalizes its core aspects: active experimentation, theorizing, and social learning. We find that agents who choose new experiments at random develop the most informative and predictive theories of the world. The agents aiming to confirm, falsify theories, or resolve theoretical disagreements end up with an illusion of epistemic success: they develop promising accounts for the data they collected, while misrepresenting the ground truth that they intended to learn about. Agents experimenting in these theory-motivated ways acquire less diverse or less representative samples from the ground truth that also turn out to be easier to account for. Random data collection, on the other hand, combines virtues of diverse and representative sampling from a target scientific domain which enables cumulative development of the successful theoretical accounts of it. We suggest that randomization, already a gold standard within experiments, is also beneficial at the level of experiments themselves.
