The LHC RF and Timing Distribution Backbone is being upgraded for the HL-LHC. A White Rabbit technology solution for the generation and the distribution of the RF, similar to the system currently employed in SPS, is being considered. To verify its suitability from a phase stability perspective, an investigation was conducted on a proof-of-concept system. The requirement for the end-nodes is +/- 1 degree of the 200 MHz SPS RF frequency. The key figure of merit to check compliancy is peak-peak phase variation which must be < 28 ps. The compliance tests campaign will be described and results presented.
Pioneering physics experiments require increasingly faster data transfers and high-throughput electronics, which drives the research towards a new class of serialisers and optical links. In this framework, the DART28, a 100 Gbps radiation tolerant serialiser and driver, has been designed in 28 nm CMOS technology, submitted in April and delivered in August 2023. The development has been coupled with an FPGA based emulation, which provided an early assessment of its behaviour, a scalable system-level demonstrator and an effective evaluation tool for compatible commercial solutions. The challenges faced in this research and the architecture of both the hardware setup and the firmware will be described.
The Low -Power Gigabit Transceiver (lpGBT) is a radiation -tolerant ASIC used in highenergy physics experiments for multipurpose high-speed bidirectional serial links. Around 200,000 chips have been tested with a production test system capable of exercising the majority of the ASIC functionality to ensure its correct operation. Furthermore, specific individual qualification tests were carried out beyond the production tester limits, including radiation, multi -drop bus topology, inter -chip communication through different types of electrical links and characterization of jitter and stability of the recovered clocks. In this article, an overview of the production and qualification tests is given together with their results demonstrating the robustness and flexibility of the lpGBT.
The data link from the detectors to the back-end stage must keep up with the requirements from the upcoming generation of High Energy Physics experiments. Last year, we presented at TWEPP the investigation on the feasibility and limitation of high data-rate links based on the 4-Level Pulse-Amplitude Modulation (PAM4) technology. Commercial PAM4 technology poses strict constraints on the data rate of links which translates into a highly complex rad-hard SerDes design. Alternatively, pushing the limit of the Non-Return to Zero (NRZ) modulated signals, a line rate of up to 28 Gbps can be realized. This is less constraining thanks to the possibility of bypassing retimers of NRZ modules. As a part of theWork Package 6 of the CERN EP Research and Development programme, the feasibility as well as the availability in the telecom and datacom market of such NRZ links have been investigated. The rate of 25.65 Gbps per lane (an integer multiplication of the 40.0798MHz Bunch Clock) with NRZ modulation have been identified as the target for the next generation of detector-to-backend links. The Demonstrator ASIC for Radiation-Tolerant Transmitter in 28 nm (DART28) chip, designed at 28 nm and targeting high-radiation hardness, is currently being designed at this data rate with a custom protocol and a Reed-Solomon Forward Error Correction (FEC). A proof-of-concept on FPGA emulating the DART28 protocol has been built for early evaluation. The system uses commercially available optoelectronics transceivers and FPGA platforms to implement the DART28 data path containing a scrambler, interleaver and FEC. The VCU129 Xilinx Virtex Ultrascale+ and Intel Stratix 10 evaluation boards were used for this work. In this paper, the implementation of the demonstrator systems will be presented. The performance characteristic of these links will be discussed and the FEC performance will be compared to that of an ideal model.
The Versatile Link Plus Demonstrator Board (VLDB+) is a board designed by CERN’s EP-ESE group to provide an evaluation kit for the Versatile Link Plus ecosystem (VL+). This reference design gathers three custom and radiation hard devices, namely, the Low-Power Gigabit Transceiver (lpGBT), the Versatile Link Plus Transceiver (VTRx+) and the FEASTMP DC-DC converters. These components are common to some of the HL-LHC experiments and constitute the main elements of the board. The VLDB+ is already being extensively used by several experiments to get acquainted with the whole ecosystem in order to facilitate their final front-end system design. In this article, all the features of the board together with the external system used for its control are described. Furthermore, a few performance results are shown and a brief status of the project is presented.
With the ever-increasing amount of data produced by the high energy physics experiments, the transmission rates from the detectors to the back-end stages must keep up. To mitigate the exponential growth of the total loss due to the increased bandwidth, the 4-Level Pulse-Amplitude Modulation (PAM4) could be envisaged. Where the line rate of Non-Return to Zero (NRZ) modulated signals is capping at 28 Gbps per lane, PAM4 allows the rate of 53.125 Gbps or above in extremely high-end applications. Investigating the implementation of such links in FPGA is one of the activities carried out by the Work Package 6 of the CERN EP Research and Development programme. A proof-of-concept system of high-speed links using PAM4-53.125 Gbps has been built, based on a Xilinx Virtex evaluation platform and various commercial optoelectronics transceivers. PAM4 standard Forward Error Correction (FEC) codes have also been implemented and characterized over electrical and optical layers in terms of coding gain and latency. Finally, the telecom and datacom markets were investigated to identify development perspectives for the research and development for future links. In this paper, the performance of the proof-of-concept implementing these high-speed links is presented and the current and future challenges for an error-free communication are discussed.
The Low Power GigaBit Transceiver (lpGBT) is a radiation-tolerant Application-Specific Integrated Circuit (ASIC) designed to implement versatile high-speed bi-directional serial links in the experiments and environment of the Large Hadron Collider. With 336 programmable registers and 11 configuration pins, this ASIC is highly configurable. The current version of the chip, the lpGBTv1, is now under test. The project will then produce chips to equip the phase-2 experiment upgrades starting in 2025. More than 180000 components will be produced, tested and distributed to the users. The test involves validation of a wide variety of features and will be achieved at three different supply voltages, at −30 °C and at room temperature. Given the number of components and the complexity of the test, one of the biggest challenges was to minimize the execution time while maximizing coverage. This paper presents the new lpGBTv1 production test system, its development stages and some of the challenges which were met during its implementation.
The pixel-strip modules for the CMS Tracker Phase-2 Upgrade for the HL-LHC integrate a readout hybrid (PS-ROH) for the control and data acquisition link. This hybrid is based on the new, low power and compact gigabit transceiver (LpGBT) and the versatile transceiver VTRx+ specifically designed for the upgrade. A characterization board was first designed to qualify the design rules and the achievable timing performance of the gigabit block. This design enabled the development of the PS-ROH for the CMS Tracker PS modules. A testing setup was also developed to verify the PS-ROH performance before its integration into the PS modules.
The Large Hadron Collider (LHC) uses timing, trigger and control (TTC) system backbone to distribute the bunch clock and other critical timing signals to all the participating experiments. The clock signal is directly derived from the radio frequency (RF) driving the beams in the accelerator. The whole range of electronic systems from an ADC to high-speed transmission link works in synchronous to the clock signal and are sensitive to jitter. Throughout the clock distribution chain, high-frequency components increase the jitter. Multiple Phase-Locked Loops (PLLs) are used in the entire chain to maintain the jitter noise at a minimum level. Si5344 PLL by Silicon Labs is chosen as one of the candidate PLLs for jitter cleaning of the embedded clock in LHC gigabit serial link transmission. The article aims to highlight the qualification tests conducted to characterize the PLL component. Laboratory test setup is built to emulate the thermal variation of the LHC as in run-time condition. The present research study investigates the influence of temperature variation on PLL jitter-cleaning performance when operated in different configuration modes needed in TTC distribution chain. Stability of the PLL circuitry to lock correctly with repetitive reset assertion, power on/off cycle and marginal frequency swing about the LHC bunch clock mean frequency value is also studied.
The TTC-PON (Timing, Trigger and Control system based in Passive Optical Networks) was first investigated in 2010 in order to replace the current TTC system, responsible for delivering the bunch clock, trigger and control commands to the LHC experiments. A new prototype of the TTC-PON system is now proposed, overcoming the limitations of the formerly presented solutions. A new upstream data transmission scheme relying on longer bursts is described, together with a high-resolution calibration procedure for aligning bursts in a time division multiplexing access. An error correction scheme for downstream data transmission is also discussed.
The Versatile Link Demonstrator Board (VLDB) is the evaluation kit for the radiation-hard Optical Link ecosystem, which provides a 4.8 Gbps data transfer link for communication between front-end (FE) and back-end (BE) of the High Energy Physics experiments. It gathers the Versatile link main radiation-hard custom Application-Specific Integrated Circuits (ASICs) and modules: GBTx, GBT-SCA and VTRx/VTTx plus the FeastMP, a radiation-hard in-house designed DC-DC converter. This board is the first design allowing system-level tests of the Link with a complete interconnection of the constitutive components, allowing data acquisition, control and monitoring of FE devices with the GBT-SCA pair.
A prototype detection unit of the KM3NeT deep-sea neutrino telescope has been installed at 3500m depth 80 km offshore the Italian coast. KM3NeT in its final configuration will contain several hundreds of detection units. Each detection unit is a mechanical structure anchored to the sea floor, held vertical by a submerged buoy and supporting optical modules for the detection of Cherenkov light emitted by charged secondary particles emerging from neutrino interactions. This prototype string implements three optical modules with 31 photomultiplier tubes each. These optical modules were developed by the KM3NeT Collaboration to enhance the detection capability of neutrino interactions. The prototype detection unit was operated since its deployment in May 2014 until its decommissioning in July 2015. Reconstruction of the particle trajectories from the data requires a nanosecond accuracy in the time calibration. A procedure for relative time calibration of the photomultiplier tubes contained in each optical module is described. This procedure is based on the measured coincidences produced in the sea by the \(^{40}\)K background light and can easily be expanded to a detector with several thousands of optical modules. The time offsets between the different optical modules are obtained using LED nanobeacons mounted inside them. A set of data corresponding to 600 h of livetime was analysed. The results show good agreement with Monte Carlo simulations of the expected optical background and the signal from atmospheric muons. An almost background-free sample of muons was selected by filtering the time correlated signals on all the three optical modules. The zenith angle of the selected muons was reconstructed with a precision of about 3\(^\circ \).
Initiated in 2009 to emulate the GBTX (Gigabit Transceiver) serial link and test the first GBTX prototypes, the GBT-FPGA project is now a full library, targeting FPGAs (Field Programmable Gate Array) from Altera and Xilinx, allowing the implementation of one or several GBT links of two different types: "Standard" or "Latency-Optimized". The first major version of this IP Core was released in April 2014. This paper presents the various flavours of the GBT-FPGA kit and focuses on the challenge of providing a fixed and deterministic latency system both for clock and data recovery for all FPGA families.
A radiation-tolerant CDR/PLL ASIC has been developed for the upcoming LHC upgrades, featuring clock Frequency Multiplication (FM) and Clock and Data Recovery (CDR), showing deterministic phase and low jitter. Two FM modes have been implemented: either generating 40, 60, 120 and 240 MHz clock outputs for GBT-FPGA applications or providing 40, 80, 160 and 320 MHz clocks for TTC and e-link applications. The CDR operates with 40, 80, 160 or 320 Mbit/s data rates while always generating clocks at 40, 80, 160 and 320 MHz, regardless of the data rate. All the outputs are phase programmable with a resolution of 195 ps or 260 ps, depending on the selected mode. The ASIC has been designed using radiation-tolerant techniques in a 130 nm CMOS technology and operates at a 1.2 V supply voltage.
This work presents the software environment surrounding the GBTX. The GBTX is a high speed bidirectional ASIC, implementing radiation hard optical links for high-energy physics experiments. Having more than 300 8-bit configuration registers, it poses challenges addressed by a wide variety of software components. This paper focuses on the software used for characterization as well as radiation and production testing of the GBTX. It also highlights tools made available to the designers and users, enabling them to create customized configurations. The paper shows how storing data for the full GBTX lifecycle is planned to ensure a good quality tracking of their devices.
A new generation FPGA-based Timing-Trigger and Control (TTC) system based on emerging Passive Optical Network (PON) technology is being proposed to replace the existing off-detector TTC system used by the LHC experiments. High split ratio, dynamic software partitioning, low and deterministic latency, as well as low jitter are required. Exploiting the latest available technologies allows delivering higher capacity together with bidirectionality, a feature absent from the legacy TTC system. This article focuses on the features and capabilities of the latest TTC-PON prototype based on 10G-PON FTTH components along with some metrics characterizing its performance.
This paper presents the development of the GBTX radiation hard ASIC test bench. Developed for the LHC accelerator upgrade programs, the GBTX implements a bidirectional 4.8 Gb/s link between the radiation hard on-detector custom electronics and the off-detector systems. The test bench was used for functional testing of the GBTX and to evaluate its performance in a radiation environment, by conducting Total Ionizing Dose and Single-Event Upsets tests campaigns.
The LHCb experiment is upgrading part of its detector and the entire readout system towards a full 40 MHz readout system in order to run between five and ten times its initial design luminosity and increase its trigger efficiency. In this paper, the new timing, trigger and control distribution system for such an upgrade is reviewed with particular attention given to the distribution of the clock and timing information across the entire readout system, up to the FE and the on-detector electronics. Current ideas are here presented in terms of reliability, jitter, complexity and implementation.
The first prototype of a photo-detection unit of the future KM3NeT neutrino telescope has been deployed in the deep waters of the Mediterranean Sea. This digital optical module has a novel design with a very large photocathode area segmented by the use of 31 three inch photomultiplier tubes. It has been integrated in the ANTARES detector for in-situ testing and validation. This paper reports on the first months of data taking and rate measurements. The analysis results highlight the capabilities of the new module design in terms of background suppression and signal recognition. The directionality of the optical module enables the recognition of multiple Cherenkov photons from the same $$^{40}$$ K decay and the localisation of bioluminescent activity in the neighbourhood. The single unit can cleanly identify atmospheric muons and provide sensitivity to the muon arrival directions.