A common assumption in accelerator simulation is that magnets can be approximated using a hard-edge model, perhaps with some edge effects implemented in an impulse approximation. This is usually a good assumption but ignores details of the longitudinal variation of the magnetic fields, which makes it straightforward to implement symplectic tracking. Use of generalized gradient expansions [1, 2] provides an alternative approach that can suppress numerical deficiencies that may be present in computed or measured 3D field maps. However, the computation of the expansions is not particularly straightforward. We describe development of tools that simplify this process and allow use of such expansions in elegant [3] . We illustrate using the tools for Advanced Photon Source Upgrade simulations.
Elegant is an accelerator physics and particle-beam dynamics code widely used for modeling and design of a variety of high-energy particle accelerators and accelerator-based systems. In this paper we discuss a recently developed version of the code that can take advantage of CUDA-enabled graphics processing units (GPUs) to achieve significantly improved performance for a large class of simulations that are important in practice. The GPU version is largely defined by a framework that simplifies implementations of the fundamental kernel types that are used by Elegant: particle operations, reductions, particle loss, histograms, array convolutions and random number generation. Accelerated performance on the Titan Cray XK-7 supercomputer is approximately 6-10 times better with the GPU than all the CPU cores associated with the same node count. In addition to performance, the maintainability of the GPU-accelerated version of the code was considered a key design objective. Accuracy with respect to the CPU implementation is also a core consideration. Four different methods are used to ensure that the accelerated code faithfully reproduces the CPU results.
Efficient implementation of general-purpose particle tracking on GPUs can result in significant performance benefits to large-scale tracking simulations. This paper is an update on the current status of our work on accelerating Argonne National Lab’s particle accelerator simulation code ELEGANT using CUDA-enabled GPUs. We summarize the performance of beamline elements ported to GPU, and discuss optimization techniques for some important collective effects kernels, in particular our methods of avoiding costly thread contention. We also present preliminary results of a scaling study of the GPU-accelerated version of the code.
Large scale beam dynamics simulations can derive significant benefit from efficient implementation of generalpurpose particle tracking on GPUs. We present the latest results of our work on accelerating Argonne National Lab’s accelerator simulation code ELEGANT, using CUDAenabled GPUs. We summarize the performance of beamline elements ported to GPU, and discuss optimization techniques for some core collective effects kernels, in particular our methods of avoiding costly thread contention. We also outline briefly our testing and code validation infrastructure within ELEGANT as well as a new template meta-programming infrastructure for streamlining code development.
The Advanced Photon Source (APS) is a 7-GeV hard x-ray synchrotron light source consisting of 40 sectors. Sector 37 accommodates four radio-frequency cavities followed by a short straight section, which is set aside for the future installation of a diagnostic device. The 77.2-cm-long section of spool pieces can be isolated by two gate valves and have an independent vacuum pump. The spool pieces are normally under high vacuum condition when the total current is below 100 mA. However, at the higher current required for the APS Upgrade, rf heating causes an unacceptable rise in temperature. We analyzed this situation by wakefield simulation, which led to a new chamber design. Proper fabrication and careful installation with twelve thermocouples ensured a temperature rise under 40-50°C at 100 mA. A brief thermal analysis showed that the present observed temperature rise in the new chamber is mainly due to the resistive wall.
elegant is an open-source accelerator code that has been in use and development for approximately two decades. In that time, it has evolved from a graduate student project with a narrow purpose to a general code for the design and modeling of linacs and storage rings. elegant continues to evolve, thanks in no small part to suggestions from users. elegant has seen extensive application to modeling of linacs, particularly for applications related to free-electron lasers and energy recovery linacs. Recent developments have emphasized both linac and storage-ring-related enhancements, along with parallelization. In this paper, we briefly review the features of elegant and its program suite. We then describe some of the recent progress made in the ongoing development of elegant. We also discuss several noteworthy applications and directions for future work.
The BSP-100 beam position monitor (BPM) was commissioned and installed at the Advanced Photon Source (APS) in a fraction of the ring as an upgrade to the present turn-by-turn BPMs. Keeping the same rf front end of the present BPMs, the BSP-100 BPM adds a high-speed analog-to-digital converter and uses a field-programmable gate array (FPGA) to perform the signal processing. The main advantage of the new system is a much better signalto-noise ratio as all the bunches in the stored beam can now be (selectively) sampled each turn. The implementation requires a much more complex timing control. We report on the high-level software that controls, saves, restores, and compares the timing of the BSP-100 BPM. This software uses Tcl/Tk for the graphical user interface, the SDDS Toolkit for data processing, and SDDS-EPICS compliant tools for saving and restoring.
The triggering of wave-breaking in a three-dimensional laser plasma wake (bubble) is investigated. The Coulomb potential from a nanowire is used to disturb the wake field to initialize the wave-breaking. The electron acceleration becomes more stable and the laser power needed for self-trapping is lowered. Three-dimensional particle-in-cell simulations were performed. Electrons with a charge of about 100pC can be accelerated stably to energy about 170MeV with a laser energy of 460mJ. The first step towards tailoring the electron beam properties such as the energy, energy spread, and charge is discussed.
Operational goals for the Advanced Photon Source (APS) facility include 97% availability and a mean time between unscheduled beam losses (faults) of 70 hours, with more than 5000 user hours of scheduled beam per year. To meet this objective, our focus has been maximizing the mean time between faults (MTBF). We have made various hardware and software improvements to better operate and monitor the injector power supply systems. These improvements have been challenging to design and implement in light of the facility operating requirements but are critical to maintaining maximum reliability and availability of beam for user operations. This paper presents actions taken as well as future plans to continue improving injector power supply hardware and software to meet APS user operation goals.
A new Tcl/Tk widget has been created to display MEDM screens inside a Tcl/Tk application. Tcl/Tk parses the MEDM input files and the appropriate widgets are created and linked to the associated process variables. One advantage of this approach is that an X-Windows emulator is not required to view and manipulate the MEDM screen under a Windows operating system. Another benefit is that the MEDM screen can now be tightly integrated into a scripting language to attach higher-level logic to various process variable manipulations. Further details and examples of the new widget will be discussed.
Using sddspcas, a portable channel access server that is configured by SDDS input files, it is relatively simple to create process variables (PVs). It can be run in a standalone mode or it can be run so that the PVs are checked to ensure that they don’t conflict with other IOCs or portable channel access servers. It can also be run using the Run Control facility to prevent additional instances of the same sddspcas from being run. The SDDS configuration file provides the PV names, upper and lower limits, units, element counts if the PVs are waveforms, and the types of PVs. Valid types include various precision floats and integers as well as strings. One simple application of this program is that software developers can quickly test their code without requiring the coordination needed to update an IOC database to create PVs. Further details of the features, configuration, and applications of sddspcas will be discussed.
This paper introduces new features and programs developed to enhance various aspects of the SDDS-compliant EPICS toolkit. A new optimization program, sddsoptimize, was added to the toolkit; it employs the Simplex and 1-D scan methods and can be used for both EPICS and non-EPICS optimizations. Several new data logging programs were also developed, including a new, more flexible glitch logger that logs data before and after a glitch occurs. Another new data logger logs data every time the value of a process variable changes. With the data generated from this program, it is now possible to restore settings from any arbitrary time without the need for a snapshot of the system. Another new addition is the capability of saving and restoring waveform process variables. In addition to these new features, performance improvements have been realized in all the toolkit programs by replacing EZCA calls with low-level channel-access calls. Some of the toolkit programs have been upgraded to run on vxWorks to achieve higher performance
In recent years many changes have been made to the Advanced Photon Source (APS) linear accelerator (linac) to support multiple tasks. The primary purpose of the linac is to provide beam to fill the APS storage ring, which is done using thermionic cathode rf guns. At the same time we provide support for research projects, including a new facility that will be used for future operator training and testing of injector components. With each task requiring a different lattice and timing configuration, while at the same time using common rf systems, the complexity of operations has increased significantly with even greater demands being made on reliability and performance. In addition, personnel safety and equipment protection concerns have become more complex. We approached these challenges by developing three new subsystems: a highly automated linac operation using APS's Procedure Execution Manager (PEM) software; a new interlock system based on programmable logic controllers; and an automated S-band rf switching system. In this paper, we discuss how these developments have improved the flexibility and reliability of the APS linac, and how we intend to conduct operator training and test new injector components while maintaining storage ring injections.
At the Advanced Photon Source (APS) most of the testing of minor operational software is done during accelerator studies time. For major software changes, such as the porting of the complex workstation-based orhit control software to an EPICS IOC, much of the testing was done 'offline' on a test IOC. A configurable storage ring simulator was created in a workstation with corresponding control system records for correctors and orhit readbacks. The simulator¿s features will he described as well as the method used to develop and debug the most recent improvement of the APS orhit control software, among others. The simulator is also useful in general-purpose software testing.
Self-Describing Data Sets (SDDS) and the corresponding SDDS tools have long been used at the Advanced Photon Source (APS) and other laboratories for data storage and analysis. Various programs in the toolkit have been added or improved recently. Support for the Mac OS X operating system has been added. Automated testing scripts are being used to reduce bugs prior to software upgrades. The Java version of the SDDS Toolkit has been used to integrate standard SDDS functions into MATLAB. The fitting of generic functions to SDDS data has been improved. Conversion of array data to column data has been added to allow analysis of array data with existing programs. The display of sddsplot movies by saving plots to files for rapid playback has been improved.
The self-describing data set (SDDS) file protocol is a standardized way to store and access data and is the basis of an extensive toolkit. It is also the file protocol used for many accelerator design tools. Over the years, several of these SDDS-compliant accelerator programs (e.g., clinchor, elegant, estat, shower, and spiffe) have been developed at the Advanced Photon Source. Also, existing accelerator design tools for which the source code is available (e.g., ABCI, GENESIS, GINGER, MAFIA, and URMEL) have been converted to read and write SDDS files. As a result, we now have a capable set of accelerator codes that make use of the same data format and the same pre- and postprocessing suite. Further, the SDDS toolkit program sddsoptimize can be used around any of these tools or around a script that runs one or more of these tools. This provides the capability of very general, multicode optimization. In this paper, we discuss the capabilities of the existing SDDS-compliant accelerator codes, then provide examples of applications of these tools