BACKGROUND:The advent of Single Molecule Real-Time (SMRT) sequencing has overcome many limitations of second-generation sequencing, such as limited read lengths, PCR amplification biases. However, longer reads increase data volume exponentially and high error rates make many existing alignment tools inapplicable. Additionally, a single CPU's performance bottleneck restricts the effectiveness of alignment algorithms for SMRT sequencing. RESULTS:To address these challenges, we introduce ParaHAT, a parallel alignment algorithm for noisy long reads. ParaHAT utilizes vector-level, thread-level, process-level, and heterogeneous parallelism. We redesign the dynamic programming matrices layouts to eliminate data dependency in the base-level alignment, enabling effective vectorization. We further enhance computational speed through heterogeneous parallel technology and implement the algorithm for multi-node computing using MPI, overcoming the computational limits of a single node. CONCLUSIONS:Performance evaluations show that ParaHAT got a 10.03x speedup in base-level alignment, with a parallel acceleration ratio and weak scalability metric of 94.61 and 98.98% on 128 nodes, respectively.

Sequence alignment is the core of many bioinformatics tasks such as read mapping, genome assembly, variant detection and so on. With the advent of the third generation sequencing, classical dynamic programming-based alignment algorithms face challenges in efficiently handling these long reads. To address this issue, we present WFA-vect, a SIMD-based fast sequence alignment algorithm based on WFA. In WFA-vect, we introduce load synchronous and mask-based branch strategies to make the algorithm more suitable for vectorization. The load synchronous equalizes the load across different vector units to facilitate vectorization. The mask-based branch uses branch masking to bypass branch, avoiding pipeline hazards. To avoid binding the SIMD algorithm to specific hardware, we design a universal vectorization framework, which allows researchers to quickly port WFA-vect to other platforms without needing to understand the details of the algorithm. WFA-vect attains a peak speedup of 3.87x and 3.98x for data with error rates of 1% and 20%, respectively, compared to the scalar algorithm, while maintaining the alignment result consistent. The code and documentation of WFA-vect are publicly available at https://github.com/nudt-bioinfo/WFA-vect.

The utilization of large-scale datasets in various fields is increasing due to the advancement of big data technology. Due to limited computing resources, traditional serial frameworks are no longer efficient in processing such massive data. Furthermore, as Moore’s Law gradually loses its effect, improving program performance from the hardware level becomes increasingly challenging. Consequently, numerous parallel frameworks with distinct features and architectures have emerged, and selecting an appropriate one can enhance researchers’ performance across various tasks. This paper evaluates three prominent parallel frameworks-Spark, Ray, and MPI-and employs minimap2, a third-generation CPU-based sequence alignment tool, as the benchmark program. The experimental results are discussed comprehensively. To evaluate the three frameworks, we devised a parallel algorithm for minimap2 and implemented its parallel versions using Ray and MPI, respectively. Furthermore, we selected IMOS as the Spark version of minimap2. The experiments involved six real datasets and one simulated dataset to evaluate and compare speedup, efficiency, throughput, scalability, peak memory, latency, and load balance. The findings demonstrate that MPI outperforms Apache Spark and Ray in terms of achieving a maximum speedup of 104.019, 81.3% efficiency, 33.510 MB/s throughput, the lowest latency, and better load balance. However, MPI exhibits poor fault tolerance. Apache Spark demonstrated the second-best performance, with a speedup of 88.937, efficiency of 69.5%, throughput of 29.546 MB/s, low latency, and the best load balance. Furthermore, it exhibited good fault tolerance and benefited from a mature ecosystem. Ray achieves a speedup of 76.828, efficiency of 60.0%, and throughput of 25.009 MB/s. However, it experiences high latency fluctuations, possesses less load balance compared to the previous two frameworks, and maintains good fault tolerance. The source code and a comprehensive user manual for these parallel programs are available at https://github.com/Geehome/minimapR and https://github.com/Geehome/minimapM , respectively.

MOTIVATION:Structural variants (SVs) play an important role in genetic research and precision medicine. As existing SV detection methods usually contain a substantial number of false positive calls, approaches to filter the detection results are needed. RESULTS:We developed a novel deep learning-based SV filtering tool, CSV-Filter, for both short and long reads. CSV-Filter uses a novel multi-level grayscale image encoding method based on CIGAR strings of the alignment results and employs image augmentation techniques to improve SV feature extraction. CSV-Filter also utilizes self-supervised learning networks for transfer as classification models, and employs mixed-precision operations to accelerate training. The experiments showed that the integration of CSV-Filter with popular SV detection tools could considerably reduce false positive SVs for short and long reads, while maintaining true positive SVs almost unchanged. Compared with DeepSVFilter, a SV filtering tool for short reads, CSV-Filter could recognize more false positive calls and support long reads as an additional feature. AVAILABILITY AND IMPLEMENTATION:https://github.com/xzyschumacher/CSV-Filter.

Heterogeneous computing has been developing continuously in the field of high-performance computing because of its high performance and energy efficiency. More and more accelerators have emerged, such as GPU, FPGA, DSP, AI accelerator, and so on. Usually, the accelerator is connected to the host CPU as a peripheral device to form a tightly coupled heterogeneous computing node, and then, a parallel system is constructed by multiple nodes. This organization is computationally efficient, but not flexible. When new accelerators appear, it is difficult to join the system that has been built. At the hardware level, we create an array of accelerators and connect them to the existing system through a high-speed network. At the software level, we dynamically organize computing resources from various arrays to build a virtual heterogeneous computing node. This approach also includes a standard programming environment. Therefore, it is a more flexible, elastic, and scalable heterogeneous computing organization. In this paper, a supernode OpenCL implementation is proposed for hybrid parallel computing systems, in which virtual supernodes can be dynamically constructed between different computing arrays, and a standard OpenCL environment is implemented based on RDMA communication of high-speed interconnection, which can be combined with the system-level MPI programming environment, thereby realizing the large-scale parallel computing of the hybrid array. SNCL is compatible with existing MPI/OpenCL programs without the need for additional modifications. Experiments show that the runtime overhead of the supernode OpenCL environment is very low, and it is suitable for deploying applications with high computing density and large data scale between different arrays to utilize their computing power without affecting scalability.

Third-generation sequencing techniques have achieved major breakthroughs in sequencing long reads and speed. Continuous improvements in sequencing techniques have reduced sequencing costs, and the number of sequencing data files has shown explosive growth. In terms of sequence alignment, to deal with these high numbers and large-scale data, the conventional serial alignment method can no longer effectively meet the research requirements, therefore, it is of great importance to develop a faster, low-load, and compatible parallel alignment program. In this paper, we propose a parallel task pool algorithm based on the minimap2, a sequence alignment tool, and develop the task pool parallel alignment program based on this algorithm. We compare the program’s work with the average segmentation parallel alignment program. The results show that the task pool parallel alignment program has significant improvement in speedup, memory load, segmentation flexibility, and computational efficiency, it also has good scalability and computational stability. MinimapPool is available at https://github.com/krkrcc/MinimapPool.

With the exponential growth of biomedical knowledge in unstructured text repositories such as PubMed, it is imminent to establish a knowledge graph-style, efficient searchable and targeted database that can support the need of information retrieval from researchers and clinicians. To mine knowledge from graph databases, most previous methods view a triple in a graph (see Fig. 1) as the basic processing unit and embed the triplet element (i.e. drugs/chemicals, proteins/genes and their interaction) as separated embedding matrices, which cannot capture the semantic correlation among triple elements. To remedy the loss of semantic correlation caused by disjoint embeddings, we propose a novel approach to learn triple embeddings by combining entities and interactions into a unified representation. Furthermore, traditional methods usually learn triple embeddings from scratch, which cannot take advantage of the rich domain knowledge embedded in pre-trained models, and is also another significant reason for the fact that they cannot distinguish the differences implied by the same entity in the multi-interaction triples. In this paper, we propose a novel fine-tuning based approach to learn better triple embeddings by creating weakly supervised signals from pre-trained knowledge graph embeddings. The method automatically samples triples from knowledge graphs and estimates their pairwise similarity from pre-trained embedding models. The triples are then fed pairwise into a Siamese-like neural architecture, where the triple representation is fine-tuned in the manner bootstrapped by triple similarity scores. Finally, we demonstrate that triple embeddings learned with our method can be readily applied to several downstream applications (e.g. triple classification and triple clustering). We evaluated the proposed method on two open-source drug-protein knowledge graphs constructed from PubMed abstracts, as provided by BioCreative. Our method achieves consistent improvement in both triple classification and triple clustering tasks when compared to other state-of-the-art triple embedding methods, with an average 35% improvement of F1 score for the multi-interaction triples.

Read alignment is a basic and important task in genomic data analysis. The popularity of the third—generation sequencing technology has brought the need of sequence alignment algorithms to analyze long-read sequences with longer read length and high error rate. Moreover, the rapid growth of sequence data has also presented challenges for read alignment. To improve the ability to process large volume of sequencing reads, we developed a long-read sequence alignment algorithm MTMap on the heterogeneous processor FT-m7032. MTMap utilizes multi-level parallel technologies: firstly, we tailored the data structure for the wide vector processing units of DSP to speedup the score matrix computation. Secondly, we developed multithread parallelization for base-level alignment on each DSP cluster. Finally, we implemented multi-process parallelization between DSP clusters to fully exploit the computing power of FT-m7032. Experiments show that, MTMap achieves up to 16 times of parallel acceleration performance compared with the original algorithm under the condition of ensuring accuracy.

Single Molecule Real-Time (SMRT) sequencing is one of the popular issues in third-generation sequencing technology. Compared with next-generation sequencing technology, SMRT can detect single molecules and has much longer read lengths, which also leads to a huge increase in the amount of data. As the performance of a single CPU has reached its bottleneck, single-node computing is far from meeting the SMRT sequencing requirements. An alternative solution is parallel computing. It makes the alignment algorithm run on multiple computing nodes, thus greatly decreases the running time. The Regional Hashing-based Alignment Tool (rHAT) is a novel approach developed especially for SMRT sequencing. It has better sensitivity, improved correctness compared with existing sequence alignment tools. However, the original rHAT source can only run on a single node, which dramatically limits its performance. In this article, we developed PrHAT, a parallel sequence alignment version of rHAT. We test PrHAT on simulated and real datasets which the original rHAT used. Our results show that PrHAT reduces the computing wall-time from nearly an hour to several minutes. In the process of increasing the number of nodes from 2 to 16 on aligning large-scale datasets, PrHAT achieves speedups of 1.94–14.87x. The parallel efficiency decreases from 97% to 93%; moreover, its weak scaling remains almost unchanged. Based on PrHAT, we developed OpenPrHAT. It has a similar performance towards PrHAT, but can run on other computing devices like GPU in the platform. We expect that the implementation of PrHAT will promote the development of SMRT in third-generation sequencing technology.

The single molecule real-time sequencing technologies, such as PacBio and Nanopore, have higher throughput and produce longer reads, which promote the discovery of more structure variations that cannot be discovered by the second-generation sequencing data. However, compared with the second-generation sequencing data, the PacBio data lacks paired-end sequencing information, making traditional structure variations filter fail to process the new data. To solve this problem, this paper proposes a universal multi-tasking structure variation filtering model MSVF. MSVF adopts the CIGAR string defined in SAM format. CIGAR is not limited by sequencing technology or alignment algorithms, so MSVF is suitable for not only the second-generation but also the third-generation sequencing data. Moreover, CIGAR string preserves the complete sequence alignment information, which makes MSVF a highly precise model. Besides, MSVF uses deep learning methods, making it supports more structure variation types, including deletion and insertion. We trained and tested the models on the open-access NCBI datasets. The experiments proved that ShuffleNet, MobileNet, ResNet transfer learning models achieve better classification results on SVs task. The average AUC reaches more than 90% and the AUC of each category reach more than 87%. The accuracy and AUC of deletion and insertion structure variations were above 90% and above 92%, respectively. The code and data can be obtained at https://github.con weimingxiang/MSVF.

The development of third-generation sequencing technology has brought significant changes and influences on genomics. Compared to the second-generation sequencing methods, the third-generation technologies produce around 100 times longer reads to reveal new genomic variations that complete long-term gaps in the human reference genome. However, these reads' excessive length and high error rate severely increase the amount of data and alignment cost. The traditional data analysis platform and serial sequence alignment method can not effectively deal with large-scale long read alignment. There is a critical need for a novel data analysis platform that can deliver fast alignment of large-scale sequences to solve the problem of long read alignment. High-performance computing platforms and efficient, scalable algorithms based on these platforms have significant potential to impact sequence analysis approaches. This paper presented minimapR, a multi-level parallel long-read alignment tool based on minimap2, a popular third-generation read aligner. MinimapR is developed based on the new high-performance distributed framework Ray. Ray fully integrates with the Python environment and can be easily installed with pip. MinimapR can utilize the power of multiple computing nodes, significantly accelerating alignment speeds without sacrificing sensitivity. The minimapR tool was tested on 64 nodes and demonstrated a 50 fold increase in speed with 78 % parallel efficiency. The source code and user manual of minimapR are freely available at https://github.com/Geehome/minimapR.

BACKGROUND:VISPR is an interactive visualization and analysis framework for CRISPR screening experiments. However, it only supports the output of MAGeCK, and requires installation and manual configuration. Furthermore, VISPR is designed to run on a single computer, and data sharing between collaborators is challenging.RESULTS:To make the tool easily accessible to the community, we present VISPR-online, a web-based general application allowing users to visualize, explore, and share CRISPR screening data online with a few simple steps. VISPR-online provides an exploration of screening results and visualization of read count changes. Apart from MAGeCK, VISPR-online supports two more popular CRISPR screening analysis tools: BAGEL and JACKS. It provides an interactive environment for exploring gene essentiality, viewing guide RNA (gRNA) locations, and allowing users to resume and share screening results.CONCLUSIONS:VISPR-online allows users to visualize, explore and share CRISPR screening data online. It is freely available at http://vispr-online.weililab.org , while the source code is available at https://github.com/lemoncyb/VISPR-online .

CRISPR/Cas9-based gene editing technology has been widely used in various cells and organisms. However, the off-target effects will bring unpredictable consequences to the organism edited. One of the main obstacles to predict CRISPR/Cas9 off-target is the imbalance of the number of positive and negative samples, which puts forward a challenge for the training of traditional deep learning algorithms. In this paper, we proposed H-VAE, a hybrid variational autoencoder model with data augmentation. This model can extract more abundant sgRNA-DNA base pair matching information, and reduce the risk of overfitting. Moreover, the sample imbalance is resolved. H-VAE can make use of underlying information of training sample, extracted by VAE, to alleviate data-imbalance problem. In view of the weak ability to extract base pair matching information of existing models, a different encoding scheme based on pair encoding is proposed, which enables the model to make full use of sgRNA-DNA base pair matching information. On the Mismatch data set, compared with DeepCRISPR, the ROC-AUC and PR-AUC increased by 0.6% and 41.9%, respectively. In the new Indels data set test scenario, compared with CRISPR-Net, the ROC-AUC and PR-AUC were increased by 1.5% and 133.4% respectively. This proves that H-VAE can improve off-target prediction in various scenarios. The improvement of PR-AUC shows that H-VAE can significantly improve the effect of unbalanced classification. The experimental results demonstrate that H-VAE could achieve a better effect compared with state-of-the-art CRISPR/Cas9 off-target methods on various types of data sets. The code and data can be obtained at https://github.com/weimingxiang/H-VAE.

The rapid advances in sequencing technology have led to an explosion of sequence data. Sequence alignment is the central and fundamental problem in many sequence analysis procedure, while local alignment is often the kernel of these algorithms. Usually, Smith-Waterman algorithm is used to find the best subsequence match between given sequences. However, the high time complexity makes the algorithm time-consuming. A lot of approaches have been developed to accelerate and parallelize it, such as vector-level parallelization, thread-level parallelization, process-level parallelization, and heterogeneous acceleration, but the current researches seem unsystematic, which hinders the further research of parallelizing the algorithm. In this paper, we summarize the current research status of parallel local alignments and describe the data layout in these work. Based on the research status, we emphasize large-scale genomic comparisons. By surveying some typical alignment tools' performance, we discuss some possible directions in the future. We hope our work will provide the developers of the alignment tool with technical principle support, and help researchers choose proper alignment tools.

Sequence alignment and mutation detection are the basic steps of genomic data analysis.They are the premise of subsequent functional analysis,and the most time-consuming steps.In order to effectively deal with the massive genomic big data brought by high-throughput sequencing technology,MPI,OpenMP and other technologies to perform multi-level parallel optimization of sequence alignment algorithm and SNP detection algorithm were used.By testing on different data sets and parallel scales,the core algorithm reached more than 9x speedup,and the parallel efficiency remained above 60% in large-scale test.The improved algorithms obtain good parallel performance and scalability,that effectively improves the ability of genomic big data mutation detection.

Background The Type II clustered regularly interspaced short palindromic repeats (CRISPR) and CRISPR-associated proteins (Cas) is a powerful genome editing technology, which is more and more popular in gene function analysis. In CRISPR/Cas, RNA guides Cas nuclease to the target site to perform DNA modification. Results The performance of CRISPR/Cas depends on well-designed single guide RNA (sgRNA). However, the off-target effect of sgRNA leads to undesired mutations in genome and limits the use of CRISPR/Cas. Here, we present OffScan, a universal and fast CRISPR off-target detection tool. Conclusions OffScan is not limited by the number of mismatches and allows custom protospacer-adjacent motif (PAM), which is the target site by Cas protein. Besides, OffScan adopts the FM-index, which efficiently improves query speed and reduce memory consumption.

High-throughput genetic screening based on CRISPR/Cas9 or RNA-interference (RNAi) enables the exploration of genes associated with the phenotype of interest on a large scale. The rapid accumulation of public available genetic screening data provides a wealth of knowledge about genotype-to-phenotype relationships and a valuable resource for the systematic analysis of gene functions. Here we present CRISP-view, a comprehensive database of CRISPR/Cas9 and RNAi screening datasets that span multiple phenotypes, including in vitro and in vivo cell proliferation and viability, response to cancer immunotherapy, virus response, protein expression, etc. By 22 September 2020, CRISP-view has collected 10 321 human samples and 825 mouse samples from 167 papers. All the datasets have been curated, annotated, and processed by a standard MAGeCK-VISPR analysis pipeline with quality control (QC) metrics. We also developed a user-friendly webserver to visualize, explore, and search these datasets. The webserver is freely available at http://crispview.weililab.org.

Genome-wide screening using CRISPR coupled with nuclease Cas9 (CRISPR–Cas9) is a powerful technology for the systematic evaluation of gene function. Statistically principled analysis is needed for the accurate identification of gene hits and associated pathways. Here, we describe how to perform computational analysis of CRISPR screens using the MAGeCKFlute pipeline. MAGeCKFlute combines the MAGeCK and MAGeCK-VISPR algorithms and incorporates additional downstream analysis functionalities. MAGeCKFlute is distinguished from other currently available tools by its comprehensive pipeline, which contains a series of functions for analyzing CRISPR screen data. This protocol explains how to use MAGeCKFlute to perform quality control (QC), normalization, batch effect removal, copy-number bias correction, gene hit identification and downstream functional enrichment analysis for CRISPR screens. We also describe gene identification and data analysis in CRISPR screens involving drug treatment. Completing the entire MAGeCKFlute pipeline requires ~3 h on a desktop computer running Linux or Mac OS with R support. MAGeCKFlute is an algorithm for the analysis and visualization of CRISPR screen data. Starting from sequencing reads and an sgRNA library, the algorithm normalizes results and represents them as pathway enrichment classifications.

The adaptive immunity system in bacteria and archaea, Clustered Regularly Interspaced Short Palindromic Repeats, CRISPR-associate (CRISPR/Cas), has been adapted as a powerful gene editing tool and got a broad application in genome research field due to its ease of use and cost-effectiveness. The performance of CRISPR/Cas relies on well-designed single-guide RNA (sgRNA), so a lot of bioinformatic tools have been developed to assist the design of highly active and specific sgRNA. These tools vary in design specifications, parameters, genomes and so on. To help researchers to choose their proper tools, we reviewed various sgRNA design tools, mainly focusing on their on-target efficiency prediction model and off-target detection algorithm.

BACKGROUND:Novel sequence motifs detection is becoming increasingly essential in computational biology. However, the high computational cost greatly constrains the efficiency of most motif discovery algorithms.RESULTS:In this paper, we accelerate MEME algorithm targeted on Intel Many Integrated Core (MIC) Architecture and present a parallel implementation of MEME called MIC-MEME base on hybrid CPU/MIC computing framework. Our method focuses on parallelizing the starting point searching method and improving iteration updating strategy of the algorithm. MIC-MEME has achieved significant speedups of 26.6 for ZOOPS model and 30.2 for OOPS model on average for the overall runtime when benchmarked on the experimental platform with two Xeon Phi 3120 coprocessors.CONCLUSIONS:Furthermore, MIC-MEME has been compared with state-of-arts methods and it shows good scalability with respect to dataset size and the number of MICs. Source code: https://github.com/hkwkevin28/MIC-MEME .