MetaStore: High-Performance Metagenomic Analysis via In-Storage Computing.
CoRR(2023)
摘要
Metagenomics has led to significant advancements in many fields. Metagenomic
analysis commonly involves the key tasks of determining the species present in
a sample and their relative abundances. These tasks require searching large
metagenomic databases containing information on different species' genomes.
Metagenomic analysis suffers from significant data movement overhead due to
moving large amounts of low-reuse data from the storage system to the rest of
the system. In-storage processing can be a fundamental solution for reducing
data movement overhead. However, designing an in-storage processing system for
metagenomics is challenging because none of the existing approaches can be
directly implemented in storage effectively due to the hardware limitations of
modern SSDs. We propose MetaStore, the first in-storage processing system
designed to significantly reduce the data movement overhead of end-to-end
metagenomic analysis. MetaStore is enabled by our lightweight and cooperative
design that effectively leverages and orchestrates processing inside and
outside the storage system. Through our detailed analysis of the end-to-end
metagenomic analysis pipeline and careful hardware/software co-design, we
address in-storage processing challenges for metagenomics via specialized and
efficient 1) task partitioning, 2) data/computation flow coordination, 3)
storage technology-aware algorithmic optimizations, 4) light-weight in-storage
accelerators, and 5) data mapping. Our evaluation shows that MetaStore
outperforms the state-of-the-art performance- and accuracy-optimized software
metagenomic tools by 2.7-37.2$\times$ and 6.9-100.2$\times$, respectively,
while matching the accuracy of the accuracy-optimized tool. MetaStore achieves
1.5-5.1$\times$ speedup compared to the state-of-the-art metagenomic
hardware-accelerated tool, while achieving significantly higher accuracy.
更多查看译文
AI 理解论文
溯源树
样例
![](https://originalfileserver.aminer.cn/sys/aminer/pubs/mrt_preview.jpeg)
生成溯源树,研究论文发展脉络
Chat Paper
正在生成论文摘要