Supercomputing technology has been supporting the solution of cutting-edge scientific and complex engineering problems since its inception—serving as a comprehensive representation of the most advanced computer hardware and software technologies over a period of time. Over the course of nearly 80 years of development, supercomputing has progressed from being oriented towards computationally intensive tasks, to being oriented towards a hybrid of computationally and data-intensive tasks. Driven by the continuous development of high performance data analytics (HPDA) applications—such as big data, deep learning, and other intelligent tasks—supercomputing storage systems are facing challenges such as a sudden increase in data volume for computational processing tasks, increased and diversified computing power of supercomputing systems, and higher reliability and availability requirements. Based on this, data-intensive supercomputing, which is deeply integrated with data centers and smart computing centers, aims to solve the problems of complex data type optimization, mixed-load optimization, multi-protocol support, and interoperability on the storage system—thereby becoming the main protagonist of research and development today and for some time in the future. This paper first introduces key concepts in HPDA and data-intensive computing, and then illustrates the extent to which existing platforms support data-intensive applications by analyzing the most representative supercomputing platforms today (Fugaku, Summit, Sunway TaihuLight, and Tianhe 2A). This is followed by an illustration of the actual demand for data-intensive applications in today’s mainstream scientific and industrial communities from the perspectives of both scientific and commercial applications. Next, we provide an outlook on future trends and potential challenges data-intensive supercomputing is facing. In a word, this paper provides researchers and practitioners with a quick overview of the key concepts and developments in supercomputing, and captures the current and future data-intensive supercomputing research hotspots and key issues that need to be addressed.
[目的]东数西算场景下的数据流通策略需要综合考虑能耗、成本、时延等各类系统优化目标,本文采用深度强化学习算法实现高效的数据分级和流转策略.[方法]首先对东数西算场景下的数据要素流通系统架构和关键业务逻辑进行分析,并基于数据分类分级、数据传输、数据服务等关键流程的控制因素,融合多类业务优化目标,构建系统优化的通用数学模型,最后通过深度强化学习实现问题求解和策略优化.[结果]通过系统仿真,与多种基线算法进行对比,验证了本文方法在收敛性、系统长期收益、目标均衡等方面的优势.[局限]本文中方法对相关系统进行了合理的简化建模,在生产中需要基于现有方法,结合实际系统进行策略分析和回报函数定义,以进一步提升方法的推广能力和应用效果.[结论]东数西算等复杂场景下,综合考虑多种业务优化目标,并通过深度强化学习等算法对数据流通策略进行优化,能够在保障服务质量的基础上,有效提升系统自身性能.
《数据与计算发展前沿》(CN10-1649/TP,中英文双月刊)是由中国科学院主管、中国科学院计算机网络信息中心主办的学术性专业期刊.期刊旨在以开放、创新的前瞻姿态,聚焦数据、计算与网络的发展态势与技术前沿,推广原创技术方案、创新成果与最佳实践,促进学术交流,推动自然科学、社会科学以及各技术领域与先进信息技术的广泛交叉与深度融合,提高我国在该领域的科研应用水平.期刊为中国科学引文数据库(CSCD)来源期刊,入选CCF会刊以及中国科协计算领域高质量期刊分级目录(T3).
[目的]超大城市作为一个大规模复杂系统,具有高维全域性、动态复杂性、快速成长性的显著特征,其科学治理需要解决数据在跨领域、跨层级、跨主体等跨域场景下的高效流通问题.[方法]针对超大城市中数源海量、数据分散、场景复杂等主要挑战,本文提出数据高效跨域流通技术框架,该框架包括基于目录生成的分布式数据标识、基于数据路由的多主体跨域调度和基于算法切片的安全协同计算等关键技术.[结果]基于该框架构建了多层级、多粒度的目录链系统,面向北京市,支持了数千个分散数据源、上万类异质数据结构、千亿级实时数据的数据高效跨域流通,并在智能派单、证照验证等场景下进行了验证.[结论]该框架为面向超大城市治理的数据高效跨域流通提供了一种可行技术路径.
[目的]土壤水分是连接陆气水循环的重要媒介和载体.获取大尺度区域、长时间序列、高精度的土壤水分数据是环境遥感领域长期关注的难点和热点问题.[方法]本文简述了包括必要气候变量土壤水分(Essential Climate Variable Soil Moisture,ECV SM)数据、土壤水分产品系统(Soil Moisture Products System,SMOPS)数据、土壤水分主动被动(Soil Moisture Active Passive,SMAP)数据、遥感地表土壤水分(Remote-sensing-based Surface Soil Mois-ture,RSSSM)数据、神经网络土壤水分(Neural Network soil moisture,NNsm)数据、高分辨率中国区域土壤水分数据在内的6种国内外典型多源微波遥感融合土壤水分数据集的发展历程、研制方法和精度水平.[结果]通过分析数据产品的适用性,发现不同的微波遥感融合土壤水分产品各具特色,适用场景也有所区分.[结论]如何充分利用现有海量微波遥感、光学遥感、站点观测数据,设计高性能土壤水分模拟融合算法,实现对过去、现在乃至未来多情景模式下多尺度土壤干湿状况的精准掌控,是未来环境遥感与数据融合领域值得持续深入探究的重要科学方向.
