The proliferation of large language models (LLMs) has accelerated the adoption of agent-based data pipelines. Yet current approaches remain ad hoc, relying on fixed structures, predefined LLMs, and single execution engines, without systematic optimization across heterogeneous data sources and engines. This paper presents NOMA, a query optimization framework for multi-agent data pipelines. We argue that optimizing agentic pipelines is a fundamentally different query optimization problem, with central challenges: (i) a multi-dimensional search and objective space, where topology, model, and engine choices must be optimized jointly across latency, cost, and accuracy; (ii) a variable pipeline topology; (iii) the co-existence of diverse data models, leaving no common operator algebra; and (iv) the significant cost of executing these pipelines. Our controlled experiment over a real-world 10-agent pipeline reveals extreme variance (153x cost, 5x latency, 25
Early regional adoption is crucial for platform-enabled innovations because their viability depends on enough people within a region adopting them to generate network effects. Existing research on regional innovation diffusion explains early adoption largely through demographic, economic, infrastructural, or institutional conditions, paying limited attention to regional population psychology. We conceptualize regional psychological openness as an invisible social infrastructure that is relevant for understanding early platform adoption. To test this idea, we introduce a newly curated dataset that allows us to trace the initial development of three major platform innovations (i.e., accommodation sharing platforms, ride-sharing platforms, social media platforms) across hundreds of U.S. metropolitan regions. Across all three platform types, regions differ markedly in early adoption. We link these adoption data to large-scale geo-referenced personality information from millions of U.S. residents (N = 3,234,899). Results show that regional psychological openness consistently predicts earlier uptake even after accounting for demographic, socio-economic, social, infrastructural, geographical, and other psychological characteristics. In sum, the study advances a behavioral perspective on regional innovation adoption by integrating population psychology into explanations of platform diffusion.
Large Language Models (LLMs) are widely used in software engineering (SE) research and practice, yet their non-determinism, opaque training data, and rapidly evolving models threaten the reproducibility and replicability of empirical studies. We address this challenge through a collaborative effort of 22 researchers, presenting a taxonomy of seven study types that organizes how LLMs are used in SE research, together with eight guidelines for designing and reporting such studies. Each guideline distinguishes requirements (must) from recommendations (should) and is contextualized by the study types it applies to. Our guidelines recommend that researchers: (1) declare LLM usage and role; (2) report model versions, configurations, and customizations; (3) document the system and prompt design beyond the model; (4) report session traces, i.e., interaction logs and runtime traces; (5) use suitable baselines, benchmarks, and metrics; (6) include an open LLM as a baseline; (7) validate LLM outputs against human judgment; and (8) articulate limitations and mitigations. We complement the guidelines with an applicability matrix mapping guidelines to study types and a reporting checklist for authors and reviewers. We maintain the study types and guidelines online as a living resource for the community to use and shape ( llm-guidelines.org ).
Multi-agent systems of large language models (LLMs) are rapidly expanding across domains, introducing dynamics not captured by single-agent evaluations. Yet, existing work has mostly contrasted the behavior of a single agent with that of a collective of fixed size, leaving open a central question: how does group size shape dynamics? Here, we move beyond this dichotomy and systematically explore outcomes across the full range of group sizes. We focus on multi-agent misalignment, building on recent evidence that interacting LLMs playing a simple coordination game can generate collective biases absent in individual models. First, we show that collective bias is a deeper phenomenon than previously assessed: interaction can amplify individual biases, introduce new ones, or override model-level preferences. Second, we demonstrate that group size affects the dynamics in a non-linear way, revealing model-dependent dynamical regimes. Finally, we develop a mean-field analytical approach and show that, above a critical population size, simulations converge to deterministic predictions that expose the basins of attraction of competing equilibria. These findings establish group size as a key driver of multi-agent dynamics and highlight the need to consider population-level effects when deploying LLM-based systems at scale.
This paper explores the development of an algorithm for child welfare administration in Denmark. Based on an ethnographic study of the development process, we argue that scientists and IT developers enacted not one but multiple versions of "the" algorithm, and in this process, also engineered its multiple potential worlds. We conceptualize these as "algorithm-worlds"; specific sets of relations in which a version of the algorithm can exist and act. We illustrate three examples from our study: the algorithm as a docile tool within the world of child welfare casework; the algorithm as a data-connector within the world of public data infrastructure; and the algorithm as a data processor in the world of a legal assessment. We contend the merely foregrounding algorithms' multiplicity risks underestimating their world-making power, whereas construing algorithms as powerful without attending to their processes of becoming renders them seemingly singular and universal. By combining recent work on the multiplicity of algorithms in STS with actor-network-theory studies on "heterogeneous engineering," our approach, in turn, allows elucidating not only three different versions but also how each of these came with a distinct world of practice. We argue that these algorithm-worlds partially coexist, partially conflict or cascade.
