The Data Acquisition System of the Compact Muon Solenoid experiment at the Large Hadron Collider reads out event fragments of an average size of 2 kB from around 650 detector front-ends at a rate of up to 100 kHz. The first stage of event-building is performed by the Super-Fragment Builder employing custom-built electronics and a Myrinet optical network. It reduces the number of fragments by one order of magnitude, thereby greatly decreasing the requirements for the subsequent event-assembly stage. Back-pressure from the down-stream event-processing or variations in the size and rate of events may give rise to buffer overflows in the subdetector's front-end electronics, which would result in data corruption and would require a time-consuming re-sync procedure to recover. The Trigger-Throttling System protects against these buffer overflows. It provides fast feedback from any of the subdetector front-ends to the trigger so that the trigger can be throttled before buffers overflow. This paper reports on new performance measurements and on the recent successful integration of a scaled-down setup of the described system with the trigger and with front-ends of all major subdetectors. The on-going commissioning of the full-scale system is discussed.

The data acquisition system of the CMS experiment at the Large Hadron Collider features a two-stage event builder, which combines data from about 500 sources into full events at an aggregate throughput of 100 GByte/s. To meet the requirements, several architectures and interconnect technologies have been quantitatively evaluated. Both Gigabit Ethernet and Myrinet networks will be employed during the first run. Nearly full bi-section throughput can be obtained using a custom software driver for Myrinet based on barrel shifter traffic shaping.This paper discusses the use of Myrinet dual-port network interface cards supporting channel bonding to achieve virtual 5GBit/s links with adaptive routing to alleviate the throughput limitations associated with wormhole routing. Adaptive routing is not expected to be suitable for high-throughput event builder applications in high-energy physics. To corroborate this claim, results from the CMS event builder preseries installation at CERN are presented and the problems of wormhole routing networks are discussed.

The CMS Data Acquisition (DAQ) System relies on a purely software driven High Level Trigger (HLT) to reduce the full Level 1 accept rate of 100 kHz to approximately 100 Hz for archiving and later offline analysis. The HLT operates on the full information of events assembled by an event builder collecting detector data from the CMS front-end systems. The HLT software consists of a sequence of reconstruction and filtering modules executed on a farm of O(1000) CPUs built from commodity hardware. This paper presents the architecture of the CMS HLT, which integrates the CMS reconstruction framework in the online environment. The mechanisms to configure, control, and monitor the Filter Farm and the procedures to validate the filtering code within the DAQ environment are described.

A Software environment to describe configuration, control and test systems for data acquisition hardware devices is presented. The design follows a model that enforces a comprehensive use of an extensible markup language (XML) syntax to describe both the code and associated data. A feasibility study of this software, carried out for the CMS experiment at CERN, is also presented. This is based on a number of standalone applications for different hardware modules, and the design of a hardware management system to remotely access to these heterogeneous subsystems through a uniform web service interface.

The Data Acquisition system of the CMS experiment, at the CERN LHC collider, is designed to build 1 MB events at a sustained rate of 100 kHz and to provide sufficient computing power to filter the events by a factor of 1000. The Data to Surface (D2S) system is the first layer of the Data Acquisition interfacing the underground sub detector readout electronics to the surface Event Builder. It collects the 100 GB/s input data from a large number of front-end cards (650), implements a first stage event building by combining multiple sources into larger-size data fragments, and transports them to the surface for the full event building. The Data to Surface system can operate at the maximum rate of 2 Tbps. This paper describes the layout, reconfigurability and production validation of the D2S system which is to be installed by December 2005.

HyperDAQ was conceived to give users access to distributed data acquisition systems easily. To achieve that, we marry two well-established technologies: the World Wide Web and Peer-to-Peer systems. An embedded HTTP protocol engine turns an executable program into a browsable Web application that can reflect its internal data structures using a data serialization package. While the Web is based on static hyperlinks to destinations known at the time of page creation, HyperDAQ creates links to data content providers dynamically. Peer-to-Peer technology enables adaptive navigation from one application to another depending on the lifetime of application modules. Traditionally, distributed systems give the user a single point of access. We take a radically different approach. Every node may become an access point from which the whole system can be explored.

The High Level Trigger (HLT) system of the CMS experiment will consist of a series of reconstruction and selection algorithms designed to reduce the Level-1 trigger accept rate of 100 kHz to 100 Hz forwarded to permanent storage. The HLT operates on events assembled by an event builder collecting detector data from the CMS front-end system at full granularity and resolution. The HLT algorithms will run on a farm of commodity PCs, the filter farm, with a total expected computational power of 10/sup 6/ SpecInt95. The farm software, responsible for collecting, analyzing, and storing event data, consists of components from the data acquisition and the offline reconstruction domains, extended with the necessary glue components and implementation of interfaces between them. The farm is operated and monitored by the DAQ control system and must provide near-real-time feedback on the performance of the detector and the physics quality of data. In this paper, the architecture of the HLT farm is described, and the design of various software components reviewed. The status of software development is presented, with a focus on the integration issues. The physics and CPU performance of current reconstruction and selection algorithm prototypes is summarized in relation with projected parameters of the farm and taking into account the requirements of the CMS physics program. Results from a prototype test stand and plans for the deployment of the final system are finally discussed.

CMS has adopted S-LINK64 [1] as the standard interface between the detector front end readout and the central Data Acquisition (DAQ) system. The S-LINK64 is a specification of a FIFO-like interface. This includes mechanical descriptions of connector and daughter board format and electrical signal definition. The hardware/software package described in this paper (FEDkit) emulates the central DAQ side of this interface at the data rate required by the final DAQ system. The performance, integration with the CMS DAQ software framework, and plans for future developments for the DAQ input interface are also presented.

The data acquisition system of the CMS experiment at the Large Hadron Collider will employ an event builder which will combine data from about 500 data sources into full events at an aggregate throughput of 100 GByte/s. Several architectures and switch technologies have been evaluated for the DAQ Technical Design Report by measurements with test benches and by simulation. This paper describes studies of an EVB test-bench based on 64 PCs acting as data sources and data consumers and employing both Gigabit Ethernet and Myrinet technologies as the interconnect. In the case of Ethernet, protocols based on Layer-2 frames and on TCP/IP are evaluated. Results from ongoing studies, including measurements on throughput and scaling are presented. The architecture of the baseline CMS event builder will be outlined. The event builder is organised into two stages with intelligent buffers in between. The first stage contains 64 switches performing a first level of data concentration by building super-fragments from fragments of 8 data sources. The second stage combines the 64 super-fragments into full events. This architecture allows installation of the second stage of the event builder in steps, with the overall throughput scaling linearly with the number of switches in the second stage. Possible implementations of the components of the event builder are discussed and the expected performance of the full event builder is outlined.

XDAQ is a generic data acquisition software environment that emerged from a rich set of of use-cases encountered in the CMS experiment. They cover not the deployment for multiple sub-detectors and the operation of different processing and networking equipment as well as a distributed collaboration of users with different needs. The use of the software in various application scenarios demonstrated the viability of the approach. We discuss two applications, the tracker local DAQ system for front-end commissioning and the muon chamber validation system. The description is completed by a brief overview of XDAQ.

A simulation of the event building network of the Data Acquisition System of the CMS experiment at the Large Hadron Collider at CERN has been developed. The simulation of this highly complex system allows the validation of the system design and the optimization of its performance. The correctness of the simulation model is verified using measurements from test set-ups and a forecast for the full-scale system is made.

The Run Control and Monitor System (RCMS) of the CMS experiment is the set of hardware and software components responsible for controlling and monitoring the experiment during data-taking. It provides users with a "virtual counting room", enabling them to operate the experiment and to monitor detector status and data quality from any point in the world. This paper describes the architecture of the RCMS with particular emphasis on its scalability through a distributed collection of nodes arranged in a tree-based hierarchy. The current implementation of the architecture in a prototype RCMS used in test beam setups, detector validations and DAQ demonstrators is documented. A discussion of the key technologies used, including Web Services, and the results of tests performed with a 128-node system are presented.

The front-end electronics of the CMS experiment will be read out in parallel into approximately 650 modules which will be located in the underground counting room. The data read out will then be transported over a distance of similar to200 in to the surface counting room Where they will be received into deep buffers, the "Readout Units". The latter also provide the first step in the CMS event building process, by combining the data from multiple detector data sources into larger-size (similar to16 kB) data fragments. The second and final event-building step merges 64 such super-fragments into a full event. The first stage of the Event Builder, referred to as the Data to Surface (D2S) system is structured in a way to allow for a modular and scalable DAQ system whose performance can grow with the increasing luminosity of the LHC.

For the development of the CMS Data Acquisition (DAQ) system several test benches have been built. They are based on three hardware modules referred to as the FED kit. This article describes the architecture, applications and performance of these modules.The first module is a generic PCI card for moving data in and out of a PC containing a FPGA that can be programmed to customise the operation of the card. It is shown that the custom designed PCI interface transfers data at 500 MB/s, which is close to the theoretical bandwidth of 528 MB/s for the 66 MHz/ 64 bits PCI bus.The other two modules form a data transfer link based on LVDS technology with a maximal data throughput of 800 MB/s and a cable length of 7.5 meters.

Powerful mainstream computing equipment and the advent of affordable multi-Gigabit communication technology permit us to address data acquisition problems with clusters of inexpensive computers. Such networks typically incorporate heterogeneous platforms, real-time partitions and custom devices. We strive therefore for a software infrastructure that proficiently combines the nodes to a single, unified resource for the user. Overall requirements for such middleware are high efficiency and flexibility in configuration. Intelligent IO is an industry specification that defines a uniform messaging format and executing model for processor enabled communication equipment. Mapping this concept to a distributed computing environment and encapsulating the details of the specification into an application-programming framework allow us to provide run-time support for cluster operation. This paper gives a brief overview of the XDAQ framework that we designed and implemented at CERN for the Compact Muon Solenoid experiment’s prototype data acquisition system.

Java offers interesting opportunities for parallel computing. In particular, Java Remote Method Invocation (RMI) provides a flexible kind of remote procedure call (RPC) that supports polymorphism. Sun's RMI implementation achieves this kind of flexibility at the cost of a major runtime overhead. The goal of this article is to show that RMI can be implemented efficiently, while still supporting polymorphism and allowing interoperability with Java Virtual Machines (JVMs). We study a new approach for implementing RMI, using a compiler-based Java system called Manta. Manta uses a native (static) compiler instead of a just-in-time compiler. To implement RMI efficiently, Manta exploits compile-time type information for generating specialized serializers. Also, it uses an efficient RMI protocol and fast low-level communication protocols.A difficult problem with this approach is how to support polymorphism and interoperability. One of the consequences of polymorphism is that an RMI implementation must be able to download remote classes into an application during runtime. Manta solves this problem by using a dynamic bytecode compiler, which is capable of compiling and linking bytecode into a running application. To allow interoperability with JVMs, Manta also implements the Sun RMI protocol (i.e., the standard RMI protocol), in addition to its own protocol.We evaluate the performance of Manta using benchmarks and applications that run on a 32-node Myrinet cluster. The time for a null-RMI (without parameters or a return value) of Manta is 35 times lower than for the Sun JDK 1.2, and only slightly higher than for a C-based RPC protocol. This high performance is accomplished by pushing almost all of the runtime overhead of RMI to compile time. We study the performance differences between the Manta and the Sun RMI protocols in detail. The poor performance of the Sun RMI protocol is in part due to an inefficient implementation of the protocol. To allow a fair comparison, we compiled the applications and the Sun RMI protocol with the native Manta compiler. The results show that Manta's null-RMI latency is still eight times lower than for the compiled Sun RMI protocol and that Manta's efficient RMI protocol results in 1.8 to 3.4 times higher speedups for four out of six applications.