The EMDataResource Ligand Model Challenge aimed to assess the reliability and reproducibility of modeling ligands bound to protein and protein/nucleic-acid complexes in cryogenic electron microscopy (cryo-EM) maps determined at near-atomic (1.9-2.5 Å) resolution. Three published maps were selected as targets: E. coli beta-galactosidase with inhibitor, SARS-CoV-2 RNA-dependent RNA polymerase with covalently bound nucleotide analog, and SARS-CoV-2 ion channel ORF3a with bound lipid. Sixty-one models were submitted from 17 independent research groups, each with supporting workflow details. We found that (1) the quality of submitted ligand models and surrounding atoms varied, as judged by visual inspection and quantification of local map quality, model-to-map fit, geometry, energetics, and contact scores, and (2) a composite rather than a single score was needed to assess macromolecule+ligand model quality. These observations lead us to recommend best practices for assessing cryo-EM structures of liganded macromolecules reported at near-atomic resolution.

Despite the abundance of available software tools, optimal particle selection is still a vital issue in single-particle cryoelectron microscopy (cryo-EM). Regardless of the method used, most pickers struggle when ice thickness varies on a micrograph. IceBreaker allows users to estimate the relative ice gradient and flatten it by equalizing the local contrast. It allows the differentiation of particles from the background and improves overall particle picking performance. Furthermore, we introduce an additional parameter corresponding to local ice thickness for each particle. Particles with a defined ice thickness can be grouped and filtered based on this parameter during processing. These functionalities are especially valuable for on-the-fly processing to automatically pick as many particles as possible from each micrograph and to select optimal regions for data collection. Finally, estimated ice gradient distributions can be stored separately and used to inspect the quality of prepared samples.

Recently, there has been a dramatic improvement in the quality and quantity of data derived using cryogenic electron microscopy (cryo-EM). This is also associated with a large increase in the number of atomic models built. Although the best resolutions that are achievable are improving, often the local resolution is variable, and a significant majority of data are still resolved at resolutions worse than 3 Å. Model building and refinement is often challenging at these resolutions, and hence atomic model validation becomes even more crucial to identify less reliable regions of the model. Here, a graphical user interface for atomic model validation, implemented in the CCP-EM software suite, is presented. It is aimed to develop this into a platform where users can access multiple complementary validation metrics that work across a range of resolutions and obtain a summary of evaluations. Based on the validation estimates from atomic models associated with cryo-EM structures from SARS-CoV-2, it was observed that models typically favor adopting the most common conformations over fitting the observations when compared with the model agreement with data. At low resolutions, the stereochemical quality may be favored over data fit, but care should be taken to ensure that the model agrees with the data in terms of resolvable features. It is demonstrated that further re-refinement can lead to improvement of the agreement with data without the loss of geometric quality. This also highlights the need for improved resolution-dependent weight optimization in model refinement and an effective test for overfitting that would help to guide the refinement process.

Gag is the HIV structural precursor protein which is cleaved by viral protease to produce mature infectious viruses. Gag is a polyprotein composed of MA (matrix), CA (capsid), SP1, NC (nucleocapsid), SP2 and p6 domains. SP1, together with the last eight residues of CA, have been hypothesized to form a six-helix bundle responsible for the higher-order multimerization of Gag necessary for HIV particle assembly. However, the structure of the complete six-helix bundle has been elusive. Here, we determined the structures of both Gag in vitro assemblies and Gag viral-like particles (VLPs) to 4.2 Å and 4.5 Å resolutions using cryo-electron tomography and subtomogram averaging by emClarity. A single amino acid mutation (T8I) in SP1 stabilizes the six-helix bundle, allowing to discern the entire CA-SP1 helix connecting to the NC domain. These structures provide a blueprint for future development of small molecule inhibitors that can lock SP1 in a stable helical conformation, interfere with virus maturation, and thus block HIV-1 infection.

With many software tools available the optimal particle selection is still a vital issue in the single particle cryoEM. Regardless of the methods used, most pickers struggle when the varying ice thickness is present on the micrograph. We present IceBreaker, which allows us to estimate the relative ice gradient and flatten it based on the K-Means clustering algorithm, thus equalizing the local contrast. It allows differentiation of the particles from the background, and improves the particle pickers performance. Furthermore, a new parameter corresponding to the local ice thickness is introduced for each picked particle. Particles with a defined ice thickness can be grouped, sorted, and filtered based on this parameter during processing. Single particle 3D reconstructions can be made from particles in each ice group to access the effect of ice thickness. These functionalities are especially valuable for on-the-fly processing to automatically pick as many particles as possible from each micrograph and to select optimal ice regions for data collection. The software can be also used to evaluate the quality of the collected data, and also of already refined maps, deposited with the coordinates of the selected particles and to assess how the particles from different ice thickness areas contributed to the final map. Finally, the estimated ice gradient distributions can be stored separately and used to inspect the general quality of prepared samples.

This paper describes outcomes of the 2019 Cryo-EM Model Challenge. The goals were to (1) assess the quality of models that can be produced from cryogenic electron microscopy (cryo-EM) maps using current modeling software, (2) evaluate reproducibility of modeling results from different software developers and users and (3) compare performance of current metrics used for model evaluation, particularly Fit-to-Map metrics, with focus on near-atomic resolution. Our findings demonstrate the relatively high accuracy and reproducibility of cryo-EM models derived by 13 participating teams from four benchmark maps, including three forming a resolution series (1.8 to 3.1 Å). The results permit specific recommendations to be made about validating near-atomic cryo-EM structures both in the context of individual experiments and structure data archives such as the Protein Data Bank. We recommend the adoption of multiple scoring parameters to provide full and objective annotation and assessment of the model, reflective of the observed cryo-EM map density.

Significant technological developments and increasing scientific interest in cryogenic electron microscopy (cryo-EM) has resulted in a rapid increase in the amount of data generated by these experiments and the derived atomic models. Robust measures for the validation of 3D reconstructions and atomic models are essential for appropriate interpretation of the data. The resolution of data and availability of software tools that work across a range of resolutions often limit the quality of derived models. Hence, the final atomic model is often incomplete or contains regions where atomic positions are less reliable or incorrectly built. Extensive manual pruning and local adjustments or rebuilding are usually required to address these issues. The presented research introduces a software tool for the validation of the backbone trace of atomic models built in the cryo-EM density maps. In this study, we use the false discovery rate analysis, which can be used to segregate molecular signals from the background. Each atomic position in the model can be associated with an FDR backbone validation score, which can be used to identify potential mistraced residues. We demonstrate that the proposed validation score is complementary to existing validation metrics and is useful especially in cases where the model is built in the maps having varying local resolution. We also discuss the application of the score for automated pruning of atomic models built ab-initio during the iterative model building process in Buccaneer. We have implemented this score in the CCP-EM software suite.

Gag is the major HIV-1 structural polyprotein precursor. The Gag SP1 domain with the last residues of CA have been hypothesized to form a six-helix bundle necessary for particle assembly, but this bundle has not been fully resolved. Here, we determined the structures of complete CA-SP1 six-helix bundle connecting to the NC domain, from both in vitro Gag assemblies and viral-like particles (VLPs) carrying a T8I mutation in SP1, to near-atomic resolutions using cryoET and subtomogram averaging. The structures revealed novel densities, however distinct from IP6, inside the six-helix bundle of Gag assemblies, stabilizing the immature lattice. Interestingly, the T8I mutation impaired proteolytic cleavage of Gag at both SP1 boundaries. Our findings signify the involvement of small molecules in immature Gag assembly and provide the structural basis for development of small molecule inhibitors that stabilize SP1 helix, thus interfere with PR-mediated virus maturation.### Competing Interest StatementAbhay Kotecha is an employee of Thermo Fisher Scientific.

This is the full dataset of the 2019 Cryo-EM Map-based Model Metrics Challenge sponsored by EMDataResource (www.emdataresource.org, challenges.emdataresource.org, model-compare.emdataresource.org). The goals of this challenge were (1) to assess the quality of models that can be produced using current modeling software, (2) to check the reproducibility of modeling results from different software developers and users, and (3) compare the performance of current metrics used for evaluation of models. The focus was on near-atomic resolution maps with an innovative twist: three of four target maps formed a resolution series (1.8 to 3.1 Å) from the same specimen and imaging experiment. Tools developed in previous challenges were expanded for managing, visualizing and analyzing the 63 submitted coordinate models, and several new metrics were introduced. File Descriptions: 2019-EMDataResource-Challenge-web.pdf: Archive of News, Goals, Timeline, Targets, Modelling Instructions, Process, FAQ, Submission Instructions, Submission Summary Statistics source from the EMDR Challenges website correlation-images.tar.gz: Pairwise correlation tables for selected metric scores from the EMDR Model Compare website maps.tar.gz: The maps used for Fit-to-Map analyses in the Challenge models.tar.gz: The 63 models submitted by the modelling teams results.tar.gz: The output logs for all of the analysis methods Scores.xlsx: Scores for each model and analysis method, compiled into spreadsheet format targets.tar.gz: The reference models used in the analysis Post submission correction to the web archive PDF document: The full list of EMDataResource members on the model committee is as follows: Cathy Lawson, Andriy Kryshtafovych, Greg Pintilie, Mike Schmid, Helen Berman, Wah Chiu.

This paper describes outcomes of the 2019 Cryo-EM Map-based Model Metrics Challenge sponsored by EMDataResource ( www.emdataresource.org ). The goals of this challenge were (1) to assess the quality of models that can be produced using current modeling software, (2) to check the reproducibility of modeling results from different software developers and users, and (3) compare the performance of current metrics used for evaluation of models. The focus was on near-atomic resolution maps with an innovative twist: three of four target maps formed a resolution series (1.8 to 3.1 Å) from the same specimen and imaging experiment. Tools developed in previous challenges were expanded for managing, visualizing and analyzing the 63 submitted coordinate models, and several novel metrics were introduced. The results permit specific recommendations to be made about validating near-atomic cryo-EM structures both in the context of individual laboratory experiments and holdings of structure data archives such as the Protein Data Bank. Our findings demonstrate the relatively high accuracy and reproducibility of cryo-EM models derived from these benchmark maps by 13 participating teams, representing both widely used and novel modeling approaches. We also evaluate the pros and cons of the commonly used metrics to assess model quality and recommend the adoption of multiple scoring parameters to provide full and objective annotation and assessment of the model, reflective of the observed density in the cryo-EM map.