Our open-source Python package BoFire combines Bayesian Optimization (BO) with other design of experiments (DoE) strategies focusing on developing and optimizing new chemistry. Previous BO implementations, for example as they exist in the literature or software, require substantial adaptation for effective real-world deployment in chemical industry. BoFire provides a rich feature-set with extensive configurability and realizes our vision of fast-tracking research contributions into industrial use via maintainable open-source software. Owing to quality-of-life features like JSON-serializability of problem formulations, BoFire enables seamless integration of BO into RESTful APIs, a common architecture component for both self-driving laboratories and human-in-the-loop setups. This paper discusses the differences between BoFire and other BO implementations and outlines ways that BO research needs to be adapted for real-world use in a chemistry setting.

Bayesian Optimization is a useful tool for experiment design. Unfortunately, the classical, sequential setting of Bayesian Optimization does not translate well into laboratory experiments, for instance battery design, where measurements may come from different sources and their evaluations may require significant waiting times. Multi-fidelity Bayesian Optimization addresses the setting with measurements from different sources. Asynchronous batch Bayesian Optimization provides a framework to select new experiments before the results of the prior experiments are revealed. This paper proposes an algorithm combining multi-fidelity and asynchronous batch methods. We empirically study the algorithm behavior, and show it can outperform single-fidelity batch methods and multi-fidelity sequential methods. As an application, we consider designing electrode materials for optimal performance in pouch cells using experiments with coin cells to approximate battery performance.

Buffer solutions have tremendous importance in biological systems and in formulated products. Whilst the pH response upon acid/base addition to a mixture containing a single buffer can be described by the Henderson-Hasselbalch equation, modelling the pH response for multi-buffered poly-protic systems after acid/base addi-tion, a common task in all chemical laboratories and many industrial plants, is a challenge. Combining predictive modelling and experimental pH adjustment, we present an active machine learning (ML)-driven closed-loop optimization strategy for automating small scale batch pH adjustment relevant for complex samples (e.g., formulated products in the chemical industry). Several ML models were compared on a generated dataset of binary-buffered poly-protic systems and it was found that Gaussian processes (GP) served as the best performing models. Moreover, the implementation of transfer learning into the optimization protocol proved to be a suc-cessful strategy in making the process even more efficient. Finally, practical usability of the developed algorithm was demonstrated experimentally with a liquid handling robot where the pH of different buffered systems was adjusted, offering a versatile and efficient strategy for a pH adjustment processes.

Optimization over trained machine learning models has applications including: verification, minimizing neural acquisition functions, and integrating a trained surrogate into a larger decision-making problem. This paper formulates and solves optimization problems constrained by trained graph neural networks (GNNs). To circumvent the symmetry issue caused by graph isomorphism, we propose two types of symmetry-breaking constraints: one indexing a node 0 and one indexing the remaining nodes by lexicographically ordering their neighbor sets. To guarantee that adding these constraints will not remove all symmetric solutions, we construct a graph indexing algorithm and prove that the resulting graph indexing satisfies the proposed symmetry-breaking constraints. For the classical GNN architectures considered in this paper, optimizing over a GNN with a fixed graph is equivalent to optimizing over a dense neural network. Thus, we study the case where the input graph is not fixed, implying that each edge is a decision variable, and develop two mixed-integer optimization formulations. To test our symmetry-breaking strategies and optimization formulations, we consider an application in molecular design.

There has been a surge in interest in data-driven experimental design with applications to chemical engineering and drug manufacturing. Bayesian optimization (BO) has proven to be adaptable to such cases, since we can model the reactions of interest as expensive black-box functions. Sometimes, the cost of this black-box functions can be separated into two parts: (a) the cost of the experiment itself, and (b) the cost of changing the input parameters. In this short paper, we extend the SnAKe algorithm to deal with both types of costs simultaneously. We further propose extensions to the case of a maximum allowable input change, as well as to the multi-objective setting.

Constrained Bayesian optimization optimizes a black-box objective function subject to black-box constraints. For simplicity, most existing works assume that multiple constraints are independent. To ask, when and how does dependence between constraints help? , we remove this assumption and implement probability of feasibility with dependence (Dep-PoF) by applying multiple output Gaussian processes (MOGPs) as surrogate models and using expectation propagation to approximate the probabilities. We compare Dep-PoF and the independent version PoF. We propose two new acquisition functions incorporating Dep-PoF and test them on synthetic and practical benchmarks. Our results are largely negative: incorporating dependence between the constraints does not help much. Empirically, incorporating dependence between constraints may be useful if: (i) the solution is on the boundary of the feasible region(s) or (ii) the feasible set is very small. When these conditions are satisfied, the predictive covariance matrix from the MOGP may be poorly approximated by a diagonal matrix and the off-diagonal matrix elements may become important. Dep-PoF may apply to settings where (i) the constraints and their dependence are totally unknown and (ii) experiments are so expensive that any slightly better Bayesian optimization procedure is preferred. But, in most cases, Dep-PoF is indistinguishable from PoF.

Tree ensembles can be well-suited for black-box optimization tasks such as algorithm tuning and neural architecture search, as they achieve good predictive performance with little or no manual tuning, naturally handle discrete feature spaces, and are relatively insensitive to outliers in the training data. Two well-known challenges in using tree ensembles for black-box optimization are (i) effectively quantifying model uncertainty for exploration and (ii) optimizing over the piece-wise constant acquisition function. To address both points simultaneously, we propose using the kernel interpretation of tree ensembles as a Gaussian Process prior to obtain model variance estimates, and we develop a compatible optimization formulation for the acquisition function. The latter further allows us to seamlessly integrate known constraints to improve sampling efficiency by considering domain-knowledge in engineering settings and modeling search space symmetries, e.g., hierarchical relationships in neural architecture search. Our framework performs as well as state-of-the-art methods for unconstrained black-box optimization over continuous/discrete features and outperforms competing methods for problems combining mixed-variable feature spaces and known input constraints.

Bayesian Optimization is a very effective tool for optimizing expensive black-box functions. Inspired by applications developing and characterizing reaction chemistry using droplet microfluidic reactors, we consider a novel setting where the expense of evaluating the function can increase significantly when making large input changes between iterations. We further assume we are working asynchronously, meaning we have to select new queries before evaluating previous experiments. This paper investigates the problem and introduces 'Sequential Bayesian Optimization via Adaptive Connecting Samples' (SnAKe), which provides a solution by considering large batches of queries and preemptively building optimization paths that minimize input costs. We investigate some convergence properties and empirically show that the algorithm is able to achieve regret similar to classical Bayesian Optimization algorithms in both synchronous and asynchronous settings, while reducing input costs significantly. We show the method is robust to the choice of its single hyper-parameter and provide a parameter-free alternative.

Multivariate chemical reaction optimization involving catalytic systems is a non-trivial task due to the high number of tuneable parameters and discrete choices. Closed-loop optimization featuring active Machine Learning (ML) represents a powerful strategy for automating reaction optimization. However, the translation of chemical reaction conditions into a machine-readable format comes with the challenge of finding highly informative features which accurately capture the factors for reaction success and allow the model to learn efficiently. Herein, we compare the efficacy of different calculated chemical descriptors for a high throughput generated dataset to determine the impact on a supervised ML model when predicting reaction yield. Then, the effect of featurization and size of the initial dataset within a closed-loop reaction optimization was examined. Finally, the balance between descriptor complexity and dataset size was considered. Ultimately, tailored descriptors did not outperform simple generic representations, however, a larger initial dataset accelerated reaction optimization.

Energy systems optimization problems are complex due to strongly non-linear system behavior and multiple competing objectives, e.g. economic gain vs. environmental impact. Moreover, a large number of input variables and different variable types, e.g. continuous and categorical, are challenges commonly present in real-world applications. In some cases, proposed optimal solutions need to obey explicit input constraints related to physical properties or safety-critical operating conditions. This paper proposes a novel data-driven strategy using tree ensembles for constrained multi-objective optimization of black-box problems with heterogeneous variable spaces for which underlying system dynamics are either too complex to model or unknown. In an extensive case study comprised of synthetic benchmarks and relevant energy applications we demonstrate the competitive performance and sampling efficiency of the proposed algorithm compared to other state-of-the-art tools, making it a useful all-in-one solution for real-world applications with limited evaluation budgets.

The astrophysical interpretation of recent experimental observations of cosmic rays relies increasingly on Monte Carlo simulations of cosmic ray propagation and acceleration. Depending on the energy range of interest, several different propagation effects inside the Milky Way as well as in extragalactic space have to be taken into account when interpreting the data. With the CRPropa framework we aim to provide a toolbox for according simulations. In recent versions of CRPropa, the ballistic single particle propagation mode aiming primarily at extragalactic cosmic rays has been complemented by a solver for the differential transport equation to address propagation of galactic cosmic rays. Additionally, modules have been developed to address cosmic ray acceleration and many improvements have been added for simulations of electromagnetic secondaries. In this contribution we will give an overview of the CRPropa simulation framework with a focus on the latest improvements and highlight selected features by example applications.

Latest developments in many research fields indicate that deep learning methods have the potential to significantly improve physics analyses. They not only enhance the performance of existing algorithms but also pave the way for new measurement techniques that are not possible with conventional methods. As the computation is highly resource-intensive both dedicated hardware and software are required to obtain results in a reasonable time which poses a substantial entry barrier. We provide direct access to this technology after a revision of the internet platform VISPA to serve the needs of researches as well as students. VISPA equips its users with working conditions on remote computing resources comparable to a local computer through a standard web browser. For providing the required hardware resources for deep learning applications we extend the CPU infrastructure with a GPU cluster consisting of 10 nodes with each 2 GeForce GTX 1080 cards. Direct access through VISPA, preinstalled analysis software and a workload management system allowed us on one hand to support more than 100 participants in a workshop on deep learning and in corresponding university classes, and on the other hand to achieve significant progress in particle and astroparticle research. We present the setup of the system and report on the performance and achievements in the above mentioned usecases.

In order to interpret cosmic ray observations, detailed modeling of propagation effects invoking all important messengers is necessary. We introduce a new photon production and propagation code as an inherent part of the CRPropa 3 software framework. By implementing additional photon production channels, which are important for energies below similar to 10(18) eV, this code can be used for multi-messenger studies connecting the TeV and sub EeV energy regime and for interpreting models of ultra-high energy cosmic ray sources. We discuss the importance of the individual production channels and propagation effects and present example applications. (C) 2018 Elsevier B.V. All rights reserved.

We describe a method of reconstructing air showers induced by cosmic rays using deep learning techniques. We simulate an observatory consisting of ground-based particle detectors with fixed locations on a regular grid. The detector's responses to traversing shower particles are signal amplitudes as a function of time, which provide information on transverse and longitudinal shower properties. In order to take advantage of convolutional network techniques specialized in local pattern recognition, we convert all information to the image-like grid of the detectors. In this way, multiple features, such as arrival times of the first particles and optimized characterizations of time traces, are processed by the network. The reconstruction quality of the cosmic ray arrival direction turns out to be competitive with an analytic reconstruction algorithm. The reconstructed shower direction, energy and shower depth show the expected improvement in resolution for higher cosmic ray energy.

End Station A Test Beam (ESTB) is a beam line at SLAC using a small fraction of the bunches of the 13.6 GeV electron beam from the Linac Coherent Light Source (LCLS), restoring test beam capabilities in the large End Station A (ESA) experimental hall. ESTB will provide one of a kind test beam essential for developing accelerator instrumentation and accelerator R&D, performing particle and particle astrophysics detector research, linear collider machine and detector interface (MDI) R&D studies, development of radiation-hard detectors, and material damage studies with several distinctive features. In the past, 18 institutions participated in the ESA program at SLAC. In stage I, 4 new kicker magnets will be added to divert 5 Hz of the LCLS beam to the A-line. A new beam dump will be installed and a new Personnel Protection System (PPS) is being built in ESA. In stage II, a secondary hadron target will be installed, able to produce pions up to about 12 GeV/c at 1 particle/pulse.

A new analysis of the dataset from the Pierre Auger Observatory provides evidence for anisotropy in the arrival directions of ultra-high-energy cosmic rays on an intermediate angular scale, which is indicative of excess arrivals from strong, nearby sources. The data consist of 5514 events above 20 EeV with zenith angles up to 80 deg recorded before 2017 April 30. Sky models have been created for two distinct populations of extragalactic gamma-ray emitters: active galactic nuclei from the second catalog of hard Fermi-LAT sources (2FHL) and starburst galaxies from a sample that was examined with Fermi-LAT. Flux-limited samples, which include all types of galaxies from the Swift-BAT and 2MASS surveys, have been investigated for comparison. The sky model of cosmic-ray density constructed using each catalog has two free parameters, the fraction of events correlating with astrophysical objects and an angular scale characterizing the clustering of cosmic rays around extragalactic sources. A maximum-likelihood ratio test is used to evaluate the best values of these parameters and to quantify the strength of each model by contrast with isotropy. It is found that the starburst model fits the data better than the hypothesis of isotropy with a statistical significance of 4.0 sigma, the highest value of the test statistic being for energies above 39 EeV. The three alternative models are favored against isotropy with 2.7-3.2 sigma significance. The origin of the indicated deviation from isotropy is examined and prospects for more sensitive future studies are discussed.

A new analysis of the data set from the Pierre Auger Observatory provides evidence for anisotropy in the arrival directions of ultra-high-energy cosmic rays on an intermediate angular scale, which is indicative of excess arrivals from strong, nearby sources. The data consist of 5514 events above 20 EeV with zenith angles up to 80°recorded before 2017 April 30. Sky models have been created for two distinct populations of extragalactic gamma-ray emitters: active galactic nuclei from the second catalog of hard Fermi-LAT sources (2FHL) and starburst galaxies from a sample that was examined with Fermi-LAT. Flux-limited samples, which include all types of galaxies from the Swift-BAT and 2MASS surveys, have been investigated for comparison. The sky model of cosmic-ray density constructed using each catalog has two free parameters, the fraction of events correlating with astrophysical objects, and an angular scale characterizing the clustering of cosmic rays around extragalactic sources. A maximum-likelihood ratio test is used to evaluate the best values of these parameters and to quantify the strength of each model by contrast with isotropy. It is found that the starburst model fits the data better than the hypothesis of isotropy with a statistical significance of 4.0σ, the highest value of the test statistic being for energies above 39 EeV. The three alternative models are favored against isotropy with 2.7σ–3.2σ significance. The origin of the indicated deviation from isotropy is examined and prospects for more sensitive future studies are discussed.

Please see the PDF file for details.

(ANITA Collaboration) P. W. Gorham1, J. Nam2, S. Hoover4, P. Allison5,6 O. Banerjee5, L. Batten11, J. J. Beatty5,6, K. Belov3, D. Z. Besson7, W. R. Binns8, V. Bugaev8, P. Cao9, C. Chen2, P. Chen2, J. M. Clem9, A. Connolly5,6, B. Dailey5, C. Deaconu10, L. Cremonesi11, P. F. Dowkontt4, M. A. DuVernois1, R. C. Field12, B. D. Fox1, D. Goldstein14, J. Gordon5, C. Hast12, C. L. Hebert1, B. Hill1, K. Hughes5, R. Hupe5, M. H. Israel8, A. Javaid9, J. Kowalski1, J. Lam4, A. Ludwig10, J. G. Learned1, K. M. Liewer3, T. C. Liu2, J. T. Link1, E. Lusczek15, S. Matsuno1, B. C. Mercurio5, C. Miki1, P.Miočinović1, M. Mottram11, K. Mulrey9, C. J. Naudet3, J. Ng12, R. J. Nichol11, A. Novikov7, K. Palladino5, S. Prohira7, B. F. Rauch8, K. Reil13, J. Roberts1, M. Rosen1, B. Rotter1, J. Russell1, L. Ruckman1, D. Saltzberg4, D. Seckel9, S. Stafford5, J. Stockham7, M. Stockham7, B. Strutt12, K. Tatem1, G. S. Varner1, A. G. Vieregg10, D. Walz12, S. A. Wissel15, and F. Wu4 1University of Hawaii, Manoa, 2National Taiwan University, 3Jet Propulsion Laboratory, 4University of California, Los Angeles 5Ohio State University, 6Center for Cosmology and Particle Astrophysics, Ohio State University, 7University of Kansas, Lawrence, 8Washington University in St. Louis, 9University of Delaware, Newark, 10Department of Physics, Enrico Fermi Institute, Kavli Institute for Cosmological Physics, University of Chicago, 11University College London, 12SLAC National Accelerator Laboratory, 13University of California, Irvine, 14University of Minnesota, Minneapolis, Minnesota, 15California Polytechnic State University,

Cosmic rays are atomic nuclei arriving from outer space that reach the highest energies observed in nature. Clues to their origin come from studying the distribution of their arrival directions. Using 3×104 cosmic rays above 8×1018 electron volts, recorded with the Pierre Auger Observatory from a total exposure of 76,800 square kilometers steradian year, we report an anisotropy in the arrival directions. The anisotropy, detected at more than the 5.2σ level of significance, can be described by a dipole with an amplitude of 6.5+1.3 −0.9% towards right ascension αd = 100± 10 degrees and declination δd = −24+12 −13 degrees. That direction indicates an extragalactic origin for these ultra-high energy particles. Particles with energies ranging from below 109 eV up to beyond 1020 eV, known as cosmic rays, constantly hit the Earth’s atmosphere. The flux of these particles steeply decreases as their energy increases; for energies above 10 EeV (1 EeV ≡ 1018 eV), the flux is about one particle per km2 per year. The existence of cosmic rays with such ultra-high energies has been known for more than 50 years [1, 2], but the sites and mechanisms of their production remain a mystery. Information about their origin can be obtained from the study of the energy spectrum and the mass composition of cosmic rays. However, the most direct evidence of the location of the progenitors is expected to come from studies of the distribution of their arrival directions. Indications of possible hot spots in arrival directions for cosmic rays with energy above 50 EeV have been reported by the Pierre Auger and Telescope Array Collaborations [3, 4], but the statistical significance of these results is low. We report the observation, significant at a level of more than 5.2σ, of a large-scale anisotropy in arrival directions of cosmic rays above 8 EeV. Above 1014 eV, cosmic rays entering the atmosphere create cascades of particles (called extensive air-showers) that are sufficiently large to reach the ground. At 10 EeV, an extensive airshower (hereafter shower) contains ∼1010 particles spread over an area of ∼20 km2 in a thin disc moving close to the speed of light. The showers contain an electromagnetic component (electrons, positrons and photons) and a muonic component that can be sampled using arrays of particle detectors. Charged particles in the shower also excite nitrogen molecules in the air, producing fluorescence light that can be observed with telescopes during clear nights. The Pierre Auger Observatory, located near the city of Malargüe, Argentina, at latitude 35.2◦S, is designed to detect showers produced by primary cosmic rays above 0.1 EeV. It is a hybrid system, a combination of an array of particle detectors and a set of telescopes used to detect the fluorescence light. Our analysis is based on data gathered from 1600 water-Cherenkov detectors deployed over an area of 3000 km2 on a hexagonal grid with 1500-m spacing. Each detector contains 12 tonnes of ultrapure water in a cylindrical container, 1.2 m deep and 10 m2 in area, viewed by three 9-inch photomultipliers. A full description of the observatory, together with details of the methods used to reconstruct the arrival directions and energies of events, has been published [5]. *correspondence to: auger spokespersons@fnal.gov †The authors with their affiliations appear at the end of this article. 1 ar X iv :1 70 9. 07 32 1v 1 [ as tr oph .H E ] 2 1 Se p 20 17 It is difficult to locate the sources of cosmic rays, as they are charged particles and thus interact with the magnetic fields in our galaxy and the intergalactic medium that lies between the sources and Earth. They undergo angular deflections with amplitude proportional to their atomic number Z, to the integral along the trajectory of the magnetic field (orthogonal to the direction of propagation), and to the inverse of their energy E. At E ≈ 10 EeV, the best estimates for the mass of the particles [6] lead to a mean value for Z between 1.7 and 5. The exact number derived is dependent on extrapolations of hadronic physics, which are poorly understood because they lie well beyond the observations made at the Large Hadron Collider. Magnetic fields are not well constrained by data, but if we adopt recent models of the Galactic magnetic field [7,8], typical values of the deflections of particles crossing the Galaxy are a few tens of degrees for E/Z = 10 EeV, depending on the direction considered [9]. Extragalactic magnetic fields may also be relevant for cosmic rays propagating through intergalactic space [10]. However, even if particles from individual sources are strongly deflected, it remains possible that anisotropies in the distribution of their arrival directions will be detectable on large angular scales, provided the sources have a nonuniform spatial distribution or, in the case of a single dominant source, if the cosmic-ray propagation is diffusive [11–14]. Searches for large-scale anisotropies are conventionally made by looking for nonuniformities in the distribution of events in right ascension [15,16] because, for arrays of detectors that operate with close to 100% efficiency, the total exposure as a function of this angle is almost constant. The nonuniformity of the detected cosmic-ray flux in declination (fig. S1) imprints a characteristic nonuniformity in the distribution of azimuth angles in the local coordinate system of the array. From this distribution it becomes possible to obtain information on the three components of a dipolar model. Event observations, selection, and calibration We analyzed data recorded at the Pierre Auger Observatory between 1 January 2004 and 31 August 2016, from a total exposure of about 76,800 km2 sr year. The 1.2-m depth of the waterCherenkov detectors enabled us to record events at a useful rate out to large values of the zenith angle, θ. We selected events with θ < 80◦ enabling the declination range −90◦ < δ < 45◦ to be explored, thus covering 85% of the sky. We adopted 4 EeV as the threshold for selection; above that energy, showers falling anywhere on the array are detected with 100% efficiency [17]. The arrival directions of cosmic rays were determined from the relative arrival times of the shower front at each of the triggered detectors; the angular resolution was better than 1◦ at the energies considered here [5]. Two methods of reconstruction have been used for showers with zenith angles above and below 60◦ [17, 18]. These have to account for the effects of the geomagnetic field [17, 19] and, in the case of showers with θ < 60◦, also for atmospheric effects [20] because systematic modulations to the rates could otherwise be induced (see supplementary materials). The energy estimators for both data sets were calibrated using events detected simultaneously by the water-Cherenkov detectors and the fluorescence telescopes, with a quasi-calorimetric determination of the energy coming from the fluorescence measurements. The statistical uncertainty in the energy determination is 16% above 4 EeV and 12% above 10 EeV, whereas the systematic uncertainty on the absolute energy scale, common to both data sets, is 14% [21]. Evidence that the analyses of the events with θ < 60◦ and of those with 60◦ < θ < 80◦ are consistent with each other comes from the energy spectra determined for the two angular bands. The spectra agree within the statistical uncertainties over the energy range of interest [22]. We consider events in two energy ranges, 4 EeV < E < 8 EeV and E ≥ 8 EeV, as adopted in previous analyses (e.g., [23–25]). The bin limits follow those chosen previously in [26, 27]. The median energies for these bins are 5.0 EeV and 11.5 EeV, respectively. In earlier work [23–25], the event selection required that the station with the highest signal be surrounded by six operational detectors—a demanding condition. The number of triggered stations is greater than four for 99.2% of all events above 4 EeV and for 99.9% of events above 8 EeV, making it possible to use