The European X-ray Free Electron Laser is a research facility located close to Hamburg, offering X-ray pulses with ultra-high brilliance and femtosecond duration at megahertz repetition rates. The detection systems necessary to unlock the full scientific potential made possible by this machine poses considerable challenges both in terms of data volume and rate, as well as the interpretation of their recorded signal. To provide optimal data quality, expert and detector-specific knowledge not easily accessible to external facility users is essential, and its implementation must cope with the generated volumes. We therefore aim to perform these preparatory processing steps and offer users a dataset suitable for further analysis as the primary data product. This work describes the machinery and workflows providing this data to users in an automatic, configurable and reproducible manner, both online during the experiment, and offline for scientific analysis afterward on the way to publication.

The idea of using ultrashort X-ray pulses to obtain images of single proteins frozen in time has fascinated and inspired many. It was one of the arguments for building X-ray free-electron lasers. According to theory, the extremely intense pulses provide sufficient signal to dispense with using crystals as an amplifier, and the ultrashort pulse duration permits capturing the diffraction data before the sample inevitably explodes. This was first demonstrated on biological samples a decade ago on the giant mimivirus. Since then, a large collaboration has been pushing the limit of the smallest sample that can be imaged. The ability to capture snapshots on the timescale of atomic vibrations, while keeping the sample at room temperature, may allow probing the entire conformational phase space of macromolecules. Here we show the first observation of an X-ray diffraction pattern from a single protein, that of Escherichia coli GroEL which at 14 nm in diameter is the smallest biological sample ever imaged by X-rays, and demonstrate that the concept of diffraction before destruction extends to single proteins. From the pattern, it is possible to determine the approximate orientation of the protein. Our experiment demonstrates the feasibility of ultrafast imaging of single proteins, opening the way to single-molecule time-resolved studies on the femtosecond timescale.

The European XFEL is a megahertz repetition-rate facility producing extremely bright and coherent pulses of a few tens of femtoseconds duration. The amount of data generated in the context of user experiments can exceed hundreds of gigabits per second, resulting in tens of petabytes stored every year. These rates and volumes pose significant challenges both for facilities and users thereof. In fact, if unaddressed, extraction and interpretation of scientific content will be hindered, and investment and operational costs will quickly become unsustainable. In this article, we outline challenges and solutions in data reduction.

In this paper we introduce and discuss the EXtra-Xwiz pipeline for the semi-automated analysis of serial femtosecond crystallography data collected at the European XFEL. EXtra-Xwiz wraps the CrystFEL software suite, exposes data in a CrystFEL-compliant format, handles the interaction with the local high-performance computing cluster and simplifies certain experiment schemes such as the pump-probe one. Alongside with the integration of EXtra-Xwiz into the European XFEL ecosystem, future plans and developments are also briefly discussed.

The Collaborative Computational Project No. 4 (CCP4) is a UK-led international collective with a mission to develop, test, distribute and promote software for macromolecular crystallography. The CCP4 suite is a multiplatform collection of programs brought together by familiar execution routines, a set of common libraries and graphical interfaces. The CCP4 suite has experienced several considerable changes since its last reference article, involving new infrastructure, original programs and graphical interfaces. This article, which is intended as a general literature citation for the use of the CCP4 software suite in structure determination, will guide the reader through such transformations, offering a general overview of the new features and outlining future developments. As such, it aims to highlight the individual programs that comprise the suite and to provide the latest references to them for perusal by crystallographers around the world.

The high pulse intensity and repetition rate of the European X-ray Free-Electron Laser (EuXFEL) provide superior temporal resolution compared with other X-ray sources. In combination with MHz X-ray microscopy techniques, it offers a unique opportunity to achieve superior contrast and spatial resolution in applications demanding high temporal resolution. In both live visualization and offline data analysis for microscopy experiments, baseline normalization is essential for further processing steps such as phase retrieval and modal decomposition. In addition, access to normalized projections during data acquisition can play an important role in decision-making and improve the quality of the data. However, the stochastic nature of X-ray free-electron laser sources hinders the use of standard flat-field normalization methods during MHz X-ray microscopy experiments. Here, an online (i.e. near real-time) dynamic flat-field correction method based on principal component analysis of dynamically evolving flat-field images is presented. The method is used for the normalization of individual X-ray projections and has been implemented as a near real-time analysis tool at the Single Particles, Clusters, and Biomolecules and Serial Femtosecond Crystallography (SPB/SFX) instrument of EuXFEL.

The X-ray microscopy technique at the European X-ray free-electron laser (EuXFEL), operating at a MHz repetition rate, provides superior contrast and spatial-temporal resolution compared to typical microscopy techniques at other X-ray sources. In both online visualization and offline data analysis for microscopy experiments, baseline normalization is essential for further processing steps such as phase retrieval and modal decomposition. In addition, access to normalized projections during data acquisition can play an important role in decision-making and improve the quality of the data. However, the stochastic nature of XFEL sources hinders the use of existing flat-flied normalization methods during MHz X-ray microscopy experiments. Here, we present an online dynamic flat-field correction method based on principal component analysis of dynamically evolving flat-field images. The method is used for the normalization of individual X-ray projections and has been implemented as an online analysis tool at the Single Particles, Clusters, and Biomolecules and Serial Femtosecond Crystallography (SPB/SFX) instrument of EuXFEL.

The idea of start-to-end simulation for experiments at advanced light sources, such as x-ray free-electron laser facilities and synchrotron facilities, has been proposed for several years. Such a simulation workflow aims to enable the tracking of x-ray radiation from its source, through the beam transportation to the photon-matter interaction region, and finally, to a photon detector. Several extant software programs simulating different aspects of an experiment have been assembled into the SIMEX platform, which has enabled several insightful works on single-particle imaging. However, the platform's complicated and sometimes conflicting dependencies have hindered its adoption in other fields, and the scientifically intrinsic complication of each part of the workflow made it challenging for users to optimize all necessary parameters. To address these challenges, we have developed SimEx-Lite, a new core python interface in the SIMEX platform that utilizes optimized templates, personalized back engine installation, new data and parameter classes based on libpyvinyl. This paper introduces the architecture of SimEx-Lite and illustrates its applicability and usefulness in scientific research through selected examples at XFEL facilities.

X-ray free electron lasers deliver photon pulses that are bright enough to observe diffraction from extremely small crystals at a time scale that outruns their destruction. As crystals are continuously replaced, this technique is termed serial femtosecond crystallography (SFX). Due to its high pulse repetition rate, the European XFEL enables the collection of rich and extensive data sets, which are suited to study various scientific problems, including ultra-fast processes. The enormous data rate, data complexity, and the nature of the pixelized multimodular area detectors at the European XFEL pose severe challenges to users. To streamline the analysis of the SFX data, we developed the semiautomated pipeline EXtra-Xwiz around the established CrystFEL program suite, thereby processing diffraction patterns on detector frames into structure factors. Here we present EXtra-Xwiz, and we introduce its architecture and use by means of a tutorial. Future plans for its development and expansion are also discussed.

Recent developments in cryogenic electron microscopy (cryo-EM) have enabled structural studies of large macromolecular complexes at resolutions previously only attainable using macromolecular crystallography. Although a number of methods can already assist in de novo building of models into high-resolution cryo-EM maps, automated and reliable map interpretation remains a challenge. Presented here is a systematic study of the accuracy of models built into cryo-EM maps using ARP/wARP. It is demonstrated that the local resolution is a good indicator of map interpretability, and for the majority of the test cases ARP/wARP correctly builds 90% of main-chain fragments in regions where the local resolution is 4.0 Å or better. It is also demonstrated that the coordinate accuracy for models built into cryo-EM maps is comparable to that of X-ray crystallographic models at similar local cryo-EM and crystallographic resolutions. The model accuracy also correlates with the refined atomic displacement parameters.

The emergence of high repetition-rate X-ray free-electron lasers (XFELs) powered by superconducting accelerator technology enables the measurement of significantly more experimental data per day than was previously possible. The European XFEL will soon provide 27,000 pulses per second, more than two orders of magnitude more than any other XFEL. The increased pulse rate is a key enabling factor for single-particle X-ray diffractive imaging, which relies on averaging the weak diffraction signal from single biological particles. Taking full advantage of this new capability requires that all experimental steps, from sample preparation and delivery to the acquisition of diffraction patterns, are compatible with the increased pulse repetition rate. Here, we show that single-particle imaging can be performed using X-ray pulses at megahertz repetition rates. The obtained results pave the way towards exploiting high repetition-rate X-ray free-electron lasers for single-particle imaging at their full repetition rate.

The performance of automated protein model building usually decreases with resolution, mainly owing to the lower information content of the experimental data. This calls for a more elaborate use of the available structural information about macromolecules. Here, a new method is presented that uses structural homologues to improve the quality of protein models automatically constructed using ARP/wARP. The method uses local structural similarity between deposited models and the model being built, and results in longer main-chain fragments that in turn can be more reliably docked to the protein sequence. The application of the homology-based model extension method to the example of a CFA synthase at 2.7 Å resolution resulted in a more complete model with almost all of the residues correctly built and docked to the sequence. The method was also evaluated on 1493 molecular-replacement solutions at a resolution of 4.0 Å and better that were submitted to the ARP/wARP web service for model building. A significant improvement in the completeness and sequence coverage of the built models has been observed.

The remote web service for macromolecular model building using ARP/wARP [1] was established in 2004 and has had over 5,000 users.Since 2017 it has undergone a complete redesign and now offers the use of all modules of the latest ARP/wARP software version 8.0 for crystallographic model building, including interpretation of cryoelectron microscopy maps.The new web service includes advanced options for real-time monitoring, re-running computational tasks with modified parameters and a comparison of the results.The redesigned web service is gaining popularity: 900 remote users ran 9,400 model-building tasks in 2018.92% of the tasks follow a molecular replacement-like scenario where some initial model is already available.Recent developments implemented in ARP/wARP enable to build atomic structures not only in maps of a medium-to-high resolution but also within 3.0-4.0Å resolution range.Half of the users operate Mac and Linux computers and 30% use Windows.A considerable portion of users (20%) submit or monitor tasks from tablets and smartphones.The ARP/wARP web service is gradually integrated into virtual frameworks [2].Most of the tasks (58%) are submitted via the dedicated web interface.Other model-building tasks are submitted to the web service automatically from the CCP4i interface and the molecular replacement pipelines Balbes, MoRDa and MrBump.The users also have an option to direct the built ARP/wARP model to PDB-REDO.

In macromolecular crystallography the initial phases obtained by experimental phasing or molecular replacement may not always be sufficiently accurate to produce an interpretable density map.Additional phase improvement steps using density modification and/or model refinement approaches may be required.Given the complexity of the phase space to be sampled, heuristic global optimisation techniques based on genetic algorithms may have their own advantages.A sampling of the phase space and phase optimisation using genetic algorithms has been attempted and has produced promising results for reflections at low resolution of the data and for the use of the third moment of density distribution, skewness, as a target function [1,2].Here we present a phase optimisation approach using a genetic algorithm with several characteristics of the density map as a target function.For two selected test cases with X-ray data to 2.5 Å resolution, we observe the development of the density histogram with a gradual shift towards the histogram of the map computed from the refined and deposited model.

The West-Life project (https://about.west-life.eu/) is a Horizon 2020 project funded by the European Commission to provide data processing and data management services for the international community of structural biologists, and in particular to support integrative experimental approaches within the field of structural biology. It has developed enhancements to existing web services for structure solution and analysis, created new pipelines to link these services into more complex higher-level workflows, and added new data management facilities. Through this work it has striven to make the benefits of European e-Infrastructures more accessible to life-science researchers in general and structural biologists in particular.

The paper presents comparative data on the impact of force-fields AMBER (ff03, ff99SB, and ff96) on the results of molecular dynamics experiments with dimeric molecules formed by ion-peptide NH2-(RADA)4-COOH in the β-conformation at two temperatures (300K and 320K). It is shown that an MD simulation in explicit water environment is the most informative approach. The use of different force-fields has a significant influence on the stability of the initial molecular conformation of the peptide over time. Finally, the simulation in ff99SB environment provides significant stability of antiparallel β-structure of the dimer at 300K, while ff96 not only ensures the highest stability of the initial b-peptide conformation at higher temperatures, but also enhances the retention of antiparallel β-conformation, which determines the ability of NH2-(RADA)4-COOH peptides to self-organization.

We analyzed numerically computed velocity autocorrelation functions and generalized frequency spectra of charge distribution in homogeneous DNA sequences at finite temperature. The autocorrelation function and generalized frequency spectrum (frequency-dependent diffusion coefficient) are phenomenologically introduced based on the functional of mean-square displacement of the charge in DNA. The charge transfer in DNA was modeled in the framework of the semi-classical Holstein model. In this model, DNA is represented by a chain of oscillators placed into thermostat at a given temperature that is provided by the additional Langevin term. Correspondence to the real DNA is provided by choice of the force parameters, which are calculated with quantum-chemical methods. We computed the diffusion coefficient for all homogenous DNA chains with respect to the temperature and found a special scaling of independent variables that the temperature dependence of the diffusion coefficient for different homogenous DNA is almost similar. Our calculations suggest that for all the sequences, only one parameter of the system is mainly responsible for the charge kinetics. The character of individual motions contributing to the charge mobility and temperature-dependent regimes of charge distribution is determined.

The dynamics of a system for different types of polarons, i.e., in polythymine nucleotides (large-radius polaron), in polyadenine fragments (small-radius polaron), and in polyguanine DNA (intermediate case) at different thermostat temperatures are calculated using the semi-classical Holstein model. The temperature dependences of the thermodynamic equilibrium values of the total energy, the energy of an excess charge, and the electronic heat capacity have been obtained. For all polaron types, the peak of the electronic heat capacity dependence on temperature separates two modes (polaron and delocalized state). The electronic part of the energy is estimated in the high-temperature limit. In all cases, the electron heat capacity at high temperatures decreases in inverse proportion to the square of the temperature.

The temperature dependence of a charge mobility in a model DNA based on a Holstein Hamiltonian is calculated for four types of homogeneous sequences It has turned out that upon rescaling all four types are quite similar. Two types of rescaling, i.e., those for low and intermediate temperatures, are found. The curves obtained are approximated on a logarithmic scale by cubic polynomials. We believe that for model homogeneous biopolymers with parameters close to the designed ones, one can assess the value of the charge mobility without carrying out resource-intensive direct simulation, just by using a suitable approximating function.