The CMS Phase-2 upgrade of the CMS Data Acquisition (DAQ) system is based on the DAQ and Timing Hub (DTH) board. The DTH serves as an interface, connecting sub-detector front-end electronics to the DAQ, as well as timing and trigger control and distribution system. This work focuses on the DAQ functionality of the DTH, detailing the current status of the project, including the DAQ firmware and control software. First integration with Phase-2 sub-systems and future plans are presented.

To achieve better computational efficiency and exploit a wider range of computing resources, the CMS software framework (CMSSW) has been extended to offload part of the physics reconstruction to NVIDIA GPUs. To support additional back-ends, as well to avoid the need to write, validate and maintain a separate implementation of the reconstruction algorithms for each back-end, CMS has adopted the alpaka performance portability library. Alpaka (Abstraction Library for Parallel Kernel Acceleration) is a header-only C++ library that provides performance portability across different back-ends, abstracting the underlying levels of parallelism. It supports serial and parallel execution on CPUs, and extremely parallel execution on NVIDIA, AMD and Intel GPUs. This contribution will show how alpaka is used in the CMS software to develop and maintain a single code base; to use different toolchains to build the code for each supported back-end, and link them into a single application; to seamlessly select the best backend at runtime, and implement portable reconstruction algorithms that run efficiently on CPUs and GPUs from different vendors. It will describe the validation and deployment of the alpaka-based implementation in the CMS High Level Trigger, and highlight how it achieves near-native performance.

The event builder in the Data Acquisition System (DAQ) of the CMS experiment at the CERN Large Hadron Collider (LHC) is responsible for assembling events at a rate of 100 kHz during the current LHC run 3, and up to750 kHz for the upcoming High Luminosity LHC, scheduled to start in 2029. Both the current and future DAQ architectures leverage on state-of-the-art network technologies, employing Ethernet switches capable of supporting RDMA over Converged Ethernet (RoCE) protocols. The DAQ Front-end hardware is custom-designed, utilizing a reduced TCP/IP protocol implemented in FPGA for reliable data transport between custom electronics and commercial computing hardware. An alternative architecture for the event builder, known as the Super-Fragment Builder (SFB), is under evaluation. The SFB comprises two separate systems: the Super-Fragment Builder and the File-based Filter Farm (F3). A super-fragment consists of the event data read by one or more Front-End Drivers and corresponding to the same L1 accept, and the SFB constructs multiple super-fragments corresponding to the number of Read-Unit (RU) machines in the DAQ system, storing them in local RAM disks. Subsequently, the F3 accesses super-fragments from all RU machines via the Network File System (NFS) over Ethernet and builds complete events within the High Level Trigger process. This paper describes the first prototype of the SFB and presents preliminary performance results obtained within the DAQ system for LHC Run 3.

The Online Monitoring System (OMS) at the Compact Muon Solenoid (CMS) experiment at CERN collects and combines various non-event data sources, allowing users to view, compare, and correlate real-time and historical information. The OMS data warehouse (DWH) constitutes the foundation of the Online Monitoring System (OMS) architecture within CMS, responsible for the storage and manipulation of non-event data within ORACLE databases. Using PL/SQL code, the DWH orchestrates the data aggregation and processing from several sources, inheriting and revamping code from a previous project known as Web Based Monitoring (WBM) to meet evolving requirements. The main goals of the WBM databases restructuring were: the modernization of inherited PL/SQL code, necessitating the creation of new aggregation tables and the implementation of enhancements such as standardized naming conventions, improved development workflows, and continuous integration strategies. DWH is composed of multiple Oracle schemas and integrates external PL/SQL libraries, particularly the CERN Beams Common4Oracle library, which consolidates common functionalities from various CERN Beams department databases into a unified codebase for widespread application. This article describes into the architecture and development strategies employed within the OMS data warehouse, underscoring its role in facilitating efficient data aggregation and management within the OMS project in the CMS experiment at CERN.

For the Phase-2 upgrade of the CMS experiment, the CMS central DAQ group designed and developed two custom ATCA boards. These boards provide the interfaces between the sub-detector electronics and the central CMS systems. This paper describes our experience with the chosen prototyping strategy, with a focus on the design modification choices made along the way. It concludes with a brief overview of recent firmware developments, and a look at the transition towards the full board production.

The data acquisition (DAQ) system stands as an essential component within the CMS experiment at CERN. It relies on a large network system of computers with demanding requirements on control, monitoring, configuration and high throughput communication. Furthermore, the DAQ system must accommodate various application scenarios, such as interfacing with external systems, accessing custom electronics devices for data readout, and event building. We present a versatile and highly modular programmable C++ framework designed for crafting applications tailored to various needs, facilitating development through the composition and integration of modules to achieve the desired DAQ capabilities. This framework takes advantage of reusable components and readily available off-the-shelf technologies. Applications are structured to seamlessly integrate into a containerized ecosystem, where the hierarchy of components and their aggregation is specified to form the final deployable unit to be used across multiple computers or nodes within an orchestrating environment. The utilization of the framework, along with the containerization of applications, enables coping with the complexity of implementing the CMS DAQ system by providing standardized structures and components to achieve a uniform and consistent architecture.

Level-1 Data Scouting (L1DS) is a novel data acquisition subsystem at the Compact Muon Solenoid (CMS) Level-1 Trigger (L1T) that exposes the L1T event selection data primitives for online processing at the 40 MHz bunchcrossing rate of the Large Hadron Collider (LHC), enabling unbiased and unconventional analyses. An L1DS demonstrator has been operating since Run 3, relying on a ramdisk for ephemeral storage of incoming and intermediate data, accessible by the system’s units through a Network File System (NFS). With the High-Luminosity LHC (HL-LHC) and the CMS Phase 2 upgrade projected to enhance trigger resolutions, a high-performance shared memory system is key to retain real-time processing capabilities in Run 4. For this, we leverage the emerging Compute Express Link (CXL) open standard, which provides uniform and cache-coherent memory access from heterogeneous processing units, targeting a streamlined pipeline with minimized data movement over a memory lake shared among CPUs and GPUs. In this contribution, we present the integration of CXL-compliant shared memory into the L1DS demonstrator at CMS, including an overview of our approach’s design, benefits, and limitations. Furthermore, we evaluate CXL-based L1DS performance through analyses in heterogeneous contexts, supporting a discussion of the memory lake model and its use cases for the CMS community.

The CMS detector will undergo a significant upgrade to cope with the HL-LHC instantaneous luminosity and average number of proton–proton collisions per bunch crossing (BX). The Phase-2 CMS detector will be equipped with a new Level-1 (L1) trigger system that will have access to an unprecedented level of information. Advanced reconstruction algorithms will be deployed directly on the L1 FPGA-based processors, producing reconstructed physics primitives of quasi-offline quality. The latter will be collected and processed by the Level-1 trigger Data Scouting (L1DS) system at the full bunch crossing rate. Besides providing vast amounts of data for L1 and detector monitoring, the L1DS will perform quasi-online analysis in a heterogeneous computing farm. It is expected that the study of signatures too common to fit within the L1 acceptance budget, or orthogonal to the standard physics trigger selection strategies will greatly benefit from this approach. An L1DS prototype system has been set up to operate in the current LHC Run-3, with the main goal of demonstrating the basic principle and shape the development of the Phase-2 system. The Run-3 L1DS receives trigger primitives from the Global Muon and Calorimeter Trigger, the Global Trigger decision bits and the muon segments from the Barrel Muon Track Finder. FPGA boards acquire and aggregate the synchronous trigger data streams and perform basic data reduction, before sending the trigger primitives to a set of computing nodes through 100 Gbps Ethernet connections running a simplified firmware version of the TCP/IP protocol. An Intel TBB-based DAQ software receives the TCP/IP streams and applies further processing before the ingestion of the data into a cluster of servers running the CMS reconstruction framework. The output of the computing farm are data sets in the standard CMS data analysis format. This contribution presents the Run-3 L1DS demonstrator architecture and recent physics results extracted from the collected data.

A novel Data Acquisition (DAQ) system, known as Level-1 Data Scouting (L1DS), is being introduced as part of the Level-1 (L1) trigger of the CMS experiment. The L1DS system will receive the L1 intermediate primitives from the CMS Phase-2 L1 trigger on the DAQ-800 custom boards, designed for the Phase-2 central DAQ. Firmware is being developed for this purpose on the Xilinx VCU128 board, with features similar to one half of the DAQ-800, and validated in a demonstrator for LHC Run-3. This contribution describes the firmware development in view of the target design for the DAQ-800.

The data acquisition (DAQ) of the Compact Muon Solenoid (CMS) experiment at CERN, collects data for events accepted by the Level-1 Trigger from the different detector systems and assembles them in an event builder prior to making them available for further selection in the High Level Trigger, and finally storing the selected events for offline analysis. In addition to the central DAQ providing global acquisition functionality, several separate, so-called “MiniDAQ” setups allow operating independent data acquisition runs using an arbitrary subset of the CMS subdetectors. During Run 2 of the LHC, MiniDAQ setups were running their event builder and High Level Trigger applications on dedicated resources, separate from those used for the central DAQ. This cleanly separated MiniDAQ setups from the central DAQ system, but also meant limited throughput and a fixed number of possible MiniDAQ setups. In Run 3, MiniDAQ-3 setups share production resources with the new central DAQ system, allowing each setup to operate at the maximum Level-1 rate thanks to the reuse of the resources and network bandwidth. Configuration management tools had to be significantly extended to support the synchronization of the DAQ configurations needed for the various setups. We report on the new configuration management features and on the first year of operational experience with the new MiniDAQ-3 system.

The Compact Muon Solenoid (CMS) experiment at CERN incorporates one of the highest throughput data acquisition systems in the world and is expected to increase its throughput by more than a factor of ten for High-Luminosity phase of Large Hadron Collider (HL-LHC). To achieve this goal, the system will be upgraded in most of its components. Among them, the event builder software, in charge of assembling all the data read out from the different sub-detectors, is planned to be modified from a single event builder to an orbit builder that assembles multiple events at the same time. The throughput of the event builder will be increased from the current 1.6 Tb/s to 51 Tb/s for the HL-LHC orbit builder. This paper presents preliminary network transfer studies in preparation for the upgrade. The key conceptual characteristics are discussed, concerning differences between the CMS event builder in Run 3 and the CMS Orbit Builder for the HL-LHC. For the feasibility studies, a pipestream benchmark, mimicking event-builder-like traffic has been developed. Preliminary performance tests and results are discussed.

The Online Monitoring System (OMS) at the Compact Muon Solenoid experiment (CMS) at CERN aggregates and integrates different sources of information into a central place and allows users to view, compare and correlate information. It displays real-time and historical information. The tool is heavily used by run coordinators, trigger experts and shift crews, to ensure the quality and efficiency of data taking. It provides aggregated information for many use cases including data certification. OMS is the successor of Web Based Monitoring (WBM), which was in use during Run 1 and Run 2 of the LHC. WBM started as a small tool and grew substantially over the years so that maintenance became challenging. OMS was developed from scratch following several design ideas: to strictly separate the presentation layer from the data aggregation layer, to use a well-defined standard for the communication between presentation layer and aggregation layer, and to employ widely used frameworks from outside the HEP community. A report on the experience from the operation of OMS for the first year of data taking of Run 3 in 2022 is presented.

The CMS data acquisition (DAQ) is implemented as a service-oriented architecture where DAQ applications, as well as general applications such as monitoring and error reporting, are run as self-contained services. The task of deployment and operation of services is achieved by using several heterogeneous facilities, custom configuration data and scripts in several languages. In this work, we restructure the existing system into a homogeneous, scalable cloud architecture adopting a uniform paradigm, where all applications are orchestrated in a uniform environment with standardized facilities. In this new paradigm DAQ applications are organized as groups of containers and the required software is packaged into container images. Automation of all aspects of coordinating and managing containers is provided by the Kubernetes environment, where a set of physical and virtual machines is unified in a single pool of compute resources. We demonstrate that a container-based cloud architecture provides an acrossthe-board solution that can be applied for DAQ in CMS. We show strengths and advantages of running DAQ applications in a container infrastructure as compared to a traditional application model.

The CMS Phase-2 upgrade for the HL-LHC aims at preserving and expanding the current physics capability of the experiment under extreme pileup conditions. A new tracking system incorporates a track finder processor, providing tracks to the Level-1 (L1) trigger. A new high-granularity calorimeter provides fine-grained energy deposition information in the endcap region. New front-end and back-end electronics feed the L1 trigger with high-resolution information from the barrel calorimeter and the muon systems. The upgraded L1 will be based primarily on the Xilinx Ultrascale Plus series of FPGAs, capable of sophisticated feature searches with resolution often similar to the offline reconstruction. The L1 Data Scouting system (L1DS) will capture L1 intermediate data produced by the trigger processors at the beam-crossing rate of 40 MHz, and carry out online analyses based on these limited-resolution data. The L1DS will provide fast and virtually unlimited statistics for detector diagnostics, alternative luminosity measurements, and, in some cases, calibrations. It also has the potential to enable the study of otherwise inaccessible signatures, either too common to fit in the L1 trigger accept budget or with requirements that are orthogonal to “mainstream” physics. The requirements and architecture of the L1DS system are presented, as well as some of the potential physics opportunities under study. The first results from the assembly and commissioning of a demonstrator currently being installed for LHC Run-3 are also presented. The demonstrator collects data from the Global Muon Trigger, the Layer-2 Calorimeter Trigger, the Barrel Muon Track Finder, and the Global Trigger systems of the current CMS L1. This demonstrator, as a data acquisition (DAQ) system operating at the LHC bunch-crossing rate, faces many of the challenges of the Phase-2 system, albeit with scaled-down connectivity, reduced data throughput and physics capabilities, providing a testing ground for new techniques of online data reduction and processing.

The second phase of the LHC, the high-luminosity LHC (HL-LHC), is scheduled to start in 2029, after a shutdown during which the beam intensity and focusing will be significantly upgraded. For this HL-LHC era, also the CMS detector will receive an extensive upgrade, primarily to maintain its physics performance at increasing pileup. The Phase-2 CMS Level-1 trigger rate will increase to 750 kHz, for an estimated data rate in excess of 50 Tbit/s. The Phase-2 CMS off-detector electronics will be based on the ATCA standard, with back-end boards receiving the detector data from the on-detector front-ends via custom, radiation-tolerant, optical links. The CMS Phase-2 data acquisition design tightens the integration between trigger control and data flow, extending the synchronous regime of the DAQ system. At the core of the design is the DAQ and Timing Hub, a custom ATCA hub card forming the bridge between the different, detector-specific, control, and readout electronics and the common timing, trigger, and control systems. The overall synchronization and data flow of the experiment are handled by the Trigger and Timing Control and Distribution System (TCDS). For increased flexibility during commissioning and calibration runs, the design of the Phase-2 trigger and timing distribution system breaks with the traditional distribution tree, in favor of a configurable network connecting multiple independent control units to all off-detector endpoints. In order to reduce the number of custom hardware designs required, the DAQ hardware is designed, such that it can also be used to implement the TCDS.

We present a measurement of the W boson mass in W-->e(nu) decays using 1 fb-1 of data collected with the D0 detector during Run II of the Fermilab Tevatron collider. With a sample of 499830 W-->e(nu) candidate events, we measure M(W)=80.401+/-0.043 GeV. This is the most precise measurement from a single experiment.

This paper describes recent progress on the design of the DAQ and Timing Hub, or DTH, an ATCA (Advanced Telecommunications Computing Architecture) hub board intended for the phase-2 upgrade of the CMS experiment. Prototyping was originally divided into multiple feature lines, spanning all different aspects of the DTH functionality. The second DTH prototype merges all R&D and prototyping lines into a single board, which is intended to be the production candidate. Emphasis is on the process and experience in going from the first to the second DTH prototype, which included a change of the chosen FPGA as well as the integration of a commercial networking solution.

Using a data sample of $$\sqrt{s}=13\,\text {TeV}$$ proton-proton collisions collected by the CMS experiment at the LHC in 2017 and 2018 with an integrated luminosity of $$103\text {~fb}^{-1}$$ , the $$\text {B}^{0}_{\mathrm{s}} \rightarrow \uppsi (\text {2S})\text {K}_\mathrm{S}^{0}$$ and $$\text {B}^{0} \rightarrow \uppsi (\text {2S})\text {K}_\mathrm{S}^{0} \uppi ^+\uppi ^-$$ decays are observed with significances exceeding 5 standard deviations. The resulting branching fraction ratios, measured for the first time, correspond to $${\mathcal {B}}(\text {B}^{0}_{\mathrm{s}} \rightarrow \uppsi (\text {2S})K_\mathrm{S}^{0})/{\mathcal {B}}(\text {B}^{0}\rightarrow \uppsi (\text {2S})K_\mathrm{S}^{0}) = (3.33 \pm 0.69 (\text {stat})\, \pm 0.11\,(\text {syst}) \pm 0.34\,(f_{\mathrm{s}}/f_{\mathrm{d}})) \times 10^{-2}$$ and $${\mathcal {B}}(\text {B}^{0} \rightarrow \uppsi (\text {2S})\text {K}_\mathrm{S}^{0} \uppi ^{+} \uppi ^{-})/ {\mathcal {B}}(\text {B}^{0} \rightarrow \uppsi (\text {2S})\text {K}^{0}_{\mathrm{S}}) = 0.480 \pm 0.013\,(\text {stat}) \pm 0.032\,(\text {syst})$$ , where the last uncertainty in the first ratio is related to the uncertainty in the ratio of production cross sections of $$\hbox {B}^{0}_{\mathrm{s}}$$ and $$\hbox {B}^{0}$$ mesons, $$f_{\mathrm{s}}/f_{\mathrm{d}}$$ .