The LION(evoLved sIgn mOmeNtum)optimizer was found by Google via program search,making it a unique learning-based optimization algorithm.With the incorporation of two distinct interpolations between the previous step momentum and the current step gradient,as well as the integration of decoupled weight decay,LION successfully outperforms many tradi-tional sign-based gradient descent algorithms,showing impressive performance in solving large scale deep learning problems.Although previous studies have investigated its convergence prop-erties,no research has yet provided a comprehensive analysis of its convergence rate,which is more practically relevant.Recognizing that LION can be regarded as solving a specific box-con-strained problem,this paper focuses on demonstrating its convergence to the Karush-Kuhn-Tuck-er(KKT)point at the rate ofO(√dK-1/4),where d is the problem dimension and K is the num-ber of iteration steps.Step further,we remove the constraint and establish that LION converges to the critical point of the general unconstrained problem at the same rate.This rate not only de-livers the currently optimal dependence on the problem dimension d among existing studies but also tightly matches the theoretical lower bound for nonconvex stochastic optimization algorithms with respect to the number of iterations K.Additionally,the lower bound is typically measured by the gradientℓ2norm,while the LION optimizer,as a member of the SignSGD family,usually measures the largerℓ1 norm.Since different gradient norm measures may lead to different conver-gence rate dependencies on d,to verify that our convergence rate is also optimal with respect to d,we conduct extensive experiments across various deep learning tasks.Through these experi-ments,we not only demonstrate that LION achieves lower loss and higher performance compared to standard SGD,but also empirically confirm that the gradient ℓ1/ℓ2norm ratio aligns withΘ(√d),thus proving that our convergence rate matches the theoretical lower bound with respect to d in the empirical sense.
The performance of object detection networks is often limited by the depth of the feature extraction network. Increasing network parameters may yield limited improvements in the detection system's performance. Additional careful designs of network details are necessary, but they can significantly increase training difficulty. This paper introduces a novel object detection method that utilizes generative adversarial training. Our approach focuses on minimizing the EM distance (Wasserstein distance) of the feature distribution as the primary training objective. We enhance the image features so that training of GAN (Generative Adversarial Networks) yields a feature distribution that exceeds that of the original dataset, obtaining a better feature extraction network. A new loss function is also added to the adversarial training process to ensure stable improvement of the detector. A comparative experiment conducted with the original CenterNet network on MS COCO (Microsoft Common Objects in COntext) 2017 reveals that the generative adversarial training method significantly improves the average precision for most of the examined backbone networks. Among the four backbone networks employed in the experiments, the mean improvement in network AP (Average Precision) values ranged from 0.3 to 0.9, demonstrating their success with minimal training efforts. Moreover, none of the four backbone networks experienced an increase in network parameters during inference. Experimental results indicate that the proposed architecture effectively enhances the network's feature extraction capability without compromising speed during inference.
The implementation of large-scale software systems usually depends on low-level frameworks,or third-party libraries.However,the evolution of these frameworks or libraries is independent of the upper-level applications,which brings challenges in upper-level code quality assurance.For instance,changes in framework/library code,such as adding or removing APIs and altering API semantics,may result in inconsistencies among different versions of the frame-work/library code.These inconsistencies can impact the quality of higher-level apps when devel-opers update frameworks/libraries.To address this issue,analyzing the evolution process of framework/library code APIs is essential.This analysis helps upper-level app developers swiftly choose compatible versions or adjust their code.In this context,analyzing the evolution process corresponds to constructing a framework API lifecycle model.Nowadays,existing works propose the API existence-changing model for defect detection,while not considering the influence of se-mantic changes in APIs,especially exception-related code evolution.To fill this gap,this paper adopts static analysis techniques to extract exception summary information in the framework API code,proposes a multi-step matching strategy to obtain the changing process of exceptions,and finally generates exception-aware API lifecycle models for the given framework/library project.Our approach:(1)adopts control-dependency slicing analysis to extract the conditions of the ex-ception-thrown statements;uses a parameter tracing strategy to transform exception-throwing conditions into external-variable-related preconditions;and performs inter-procedure precondition construction by a bottom-up summary-based analysis.(2)proposes the exact-matching and adap-tive-matching strategies to analyze the framework/library code changes including additions,dele-tions and modifications of APIs;generates exception-aware API lifecycle models which cover sev-en API changing types.With this approach,the API lifecycle extraction tool,JavaExP,is imple-mented,which is based on Java bytecode analysis.Compared to the state-of-the-art tool,the F1 score of exception summary information extracted by JavaExP has increased by 67％,with a re-duction in processing time by 87％.The evaluation of real-world projects shows that,compared to the exception-unaware API lifecycle modeling,JavaExP can identify 18％times more API changes.Among the 75,433 APIs under analysis,20％of APIs have changed their exception-throwing behavior at least once after API introduction.These APIs involve a total of more than 7K independent exception changes.The overall results show that the exception-aware lifecycle modeling can describe the evolution process of APIs more accurately.
Superpixel segmentation has recently seen important progress benefiting from the advances in differentiable deep learning. However, the very high-resolution superpixel segmentation still remains challenging due to the expensive memory and computation cost, making the current advanced superpixel networks fail to process. In this paper, we devise Patch Calibration Networks (PCNet), aiming to efficiently and accurately implement high-resolution superpixel segmentation. PCNet follows the principle of producing high-resolution output from low-resolution input for saving GPU memory and relieving computation cost. To recall the fine details destroyed by the down-sampling operation, we propose a novel Decoupled Patch Calibration (DPC) branch for collaboratively augment the main superpixel generation branch. In particular, DPC takes a local patch from the high-resolution images and dynamically generates a binary mask to impose the network to focus on region boundaries. By sharing the parameters of DPC and main branches, the fine-detailed knowledge learned from high-resolution patches will be transferred to help calibrate the destroyed information. To the best of our knowledge, we make the first attempt to consider the deep-learning-based superpixel generation for high-resolution cases. To facilitate this research, we build evaluation benchmarks from two public datasets and one new constructed one, covering a wide range of diversities from fine-grained human parts to cityscapes. Extensive experiments demonstrate that our PCNet can not only perform favorably against the state-of-the-arts in the quantitative results but also improve the resolution upper bound from 3K to 5K on 1080Ti GPUs.
Privacy-preserving clustering has received increasing research attention in recent years. Local differential privacy (LDP) is a privacy model without relying on trusted third parties. It plays a crucial role in distributed privacy-preserving clustering. Most previous methods focus on partition-based clustering (e.g., k-means), which necessitates many iterative interactions. Massive iterations cause a division of the privacy budget and increase the amount of individual noise, affecting the clustering utility. To address this issue, we turn to grid-based clustering and design the GC-LDP algorithm to balance the privacy and clustering utility with only three rounds of interactions. In GC-LDP, our core contribution is to develop a non-uniform grid division method via the coefficient of variation (CV), which can generate a grid structure approximating the global data distribution within two rounds of interactions. Besides, we designed a new perturbation mechanism to reduce the amount of individual noise injection. Further, a cell aggregation method is developed by exploring the relative density difference among grids to achieve multi-density clustering. Theoretical analysis and experiments on real-world datasets show that GC-LDP can obtain high-quality clustering results while satisfying local differential privacy.
Multivariate time series forecasting is very important for many applications. Many studies have been conducted for accurate and interpretable prediction methods. However, existing methods either cannot take both times series and covariates into consideration, lacking of interpretability, or ignore global trends across multivariate time series. In this paper, we aim to solve these issues. To this end, we propose a new model named TEDGE for accurate and interpretable time series prediction. In this model, we extract global trends hidden across multivariate times series to improve prediction accuracy. Meanwhile, we utilize a deep recurrent model with attention mechanism to find long-and short-term sequential patterns hidden in individual time series with interpretability. We conduct experiments on several datasets to evaluate the proposed models performance. Results demonstrate the superior performance of our proposed model.
面对"人-机-物"超融合与万物智能互联远景的现实需求,联邦算力网络充分发挥联邦学习等分布式智能技术的数据聚合优势以及"信息高铁(低熵算力网)"的计算协同优势,高效利用网络中泛在离散部署的海量数据与算力资源,从而最大化满足多种高性能、智能化计算任务需求瓶颈.同时,为建立用户泛在协作计算过程中的全生命周期安全保障和对联邦算力网络的互信任基础,差分隐私等隐私计算技术的引入成为基础性需求之一.因此,在用户自身安全和隐私不受模型逆转、梯度泄露等新兴攻击威胁的前提下,如何对大量的个性化参与用户进行有效激励,促使其积极参与并真实共享本地数据和算力,是实现联邦算力任务实际部署的关键步骤之一.然而,当前联邦算力网络的激励机制大多主要侧重于用户数据评估与公平性等计算性能相关指标研究,缺少对用户隐私需求的关注,无法有效规约隐私噪声注入过程.边缘算力节点出于自身利益考量,往往夸大隐私预算需求,造成严重的冗余精度损失.针对这一问题,本文基于改进的斯塔克伯格主从博弈模型,提出一种面向联邦算力网络的隐私计算自适应激励方法,通过两阶段的动态博弈根据分布式计算过程中隐私注入尺度进行差异化定价激励.基于反向归纳法,参与用户之间首先进行博弈均衡获取最优的本地隐私噪声预算设置策略,随后联邦参数服务器求取最优的隐私支付策略.通过理论分析,本文所提方案能够取得纳什均衡下的最优解.此外,本文还进一步对参与用户的限制条件进行了讨论,得出了用户隐私成本需求的约束上界.在EMNIST、CIFAR等公有标准数据集上的实验结果也表明,该方法相比于基于合约理论、三方博弈等理论的现有隐私激励机制,能够显著提升分布式智能协同计算任务参与各方的平均效用,在满足用户隐私需求的同时提升计算性能,大幅减少冗余损耗.
在数据库、深度学习、高效存储等数据读取性能敏感的应用场景中,数据解压性能对上层应用的服务质量有着重要影响.LZ4无损数据压缩算法具备高速解压特性,因此被广泛应用在高速解压场景中,但其运行需要消耗大量CPU资源.为减少LZ4数据解压开销,学界和业界提出了基于FPGA的LZ4数据解压加速方法.但现有方法大多采用逐字节顺序处理的计算模式,导致并行度和吞吐率存在较大不足.因此,设计实现高性能LZ4数据解压加速方法成为当前研究亟需解决的关键问题.以LZ4解压的高性能加速为目标,本文研究从多层次对LZ4解压进行并行加速设计,提出了一种基于FPGA加速的高性能LZ4数据解压方法.首先,本方法研究对LZ4序列解析过程进行并行化改进,设计实现了一个基于多字段并行解析方法的并行化序列解析器,将吞吐率从每周期单字节扩展到每周期多字节.此外,本方法对序列解析器中的高时延长度字段解析逻辑进行优化改进,设计了基于二分法的最大匹配长度快速解析方法,显著减小序列解析器的关键路径时延,使得改进后的设计时钟频率比改进前提高了约21%.其次,基于并行化序列解析器,本方法设计实现了一个高性能数据解压引擎.该引擎将序列解析与数据还原过程进行解耦设计,对解压输出数据通路进行扩展,解决了解压过程中输入输出吞吐率不匹配的问题.最后,为进一步提高吞吐率性能,本方法提出了可扩展多引擎数据解压加速器设计,并实现了一个基于CPU-FPGA架构的异构端到端数据解压加速系统原型.实验分析表明,本方法提出的数据解压引擎的每周期吞吐量是现有研究的4.1～6.8倍.该引擎实现了约1.7 GB/s的解压吞吐率,达到现有研究的2.6～6.6倍.系统原型的端到端测试和资源使用评估结果表明,本方法提出的数据解压加速系统在吞吐率和资源使用方面具备良好的可扩展性.在功耗效率方面,集成8引擎的解压加速系统原型的功效比是软件加速方法的1.6倍以上.
从时空数据中有效地提取特征表示是时空数据挖掘的基础.现有时空表示学习方法过于关注时空序列中的统计相关性,易受虚假相关性的影响,难以提取无偏、鲁棒的特征表示.本文基于结构因果模型建模时空数据的生成过程,分析观测数据中虚假相关性的成因,提出了一种基于时域偏倚校正与空域因果传递的时空因果表示学习方法.首先基于后门调整消除时域的虚假相关性,然后构建因果传递网络消除空域的虚假相关性,最后利用下游特征解码器将因果特征表示应用于下游任务中.在两个真实数据集上的实验表明,本文所提时空表示学习方法有效避免了虚假相关性的干扰,增强了模型的稳定性,使其在两个下游预测任务中对数据稀疏节点的预测误差分别降低了3%和10%.
人机对话作为人工智能的重要领域,以其方便快捷的交互特点广泛应用于任务型和闲聊型机器人等诸多商业场景,并被视为新一代人机交互的主要形式.但情绪感知与表达能力的缺乏致使人机对话技术在复杂交互场景中难以满足人们对情感交流的强烈需求.为弥补人机对话技术中情感智能的缺失,基于深度学习的情感对话响应任务被提出且已发展为对话领域中一个重要的研究方向.本文首先回顾了基于深度学习的情感对话响应任务的发展历程,其次按照任务将情感对话响应分为可控情感对话生成、共情对话响应、情绪支持、多模态情感对话生成、新任务五类.随后本文也按照常用的结构将模型进行了归类与分析,以求更细致地阐述各种结构在情感对话响应任务中的具体用法,之后介绍了常用数据与评测指标.最后本文也对模型进行了总结,并在此基础上进一步展望了该任务未来的发展方向.
量子计算机的发展对现有公钥体系的影响是实质性的.在众多后量子密码流派中,格基密码方案因其安全性、高效性等优良特点而成为了主流技术路线.密码算法在各实现平台的运行效率是后量子算法评估进程中的重要指标.FPGA(Field Programmable Gate Array)具有并行性架构,是密码体系实际部署时的重要硬件平台.近年来,后量子算法的硬件优化实现研究吸引了越来越多的关注.针对格基密码算法中计算复杂度最高、耗时最长的操作——环上多项式乘法计算,本文对主流的加速技术——数论变换技术(Number-Theoretic Transform,NTT)进行了系统研究,根据参数不同将其分为标准NTT(Standard-NTT,SNTT)、删减NTT(Truncated-NTT,TNTT)、混合NTT(Hybrid-NTT,HNTT).为了提高适用性,设计了一种统一型、可常数时间执行、支持多参数的硬件电路,支持三种NTT架构,结合Karatsuba技巧,将对应系数相乘中乘法次数减少20%.针对NTT的核心操作——蝴蝶变换,设计了一种紧凑型、低时延的电路结构,可实现正向NTT、对应系数相乘、逆向NTT功能,同时支持Cooley-Tukey和Gentleman-Sande两种结构;提出一种新的系数访存模式,采用"交叉存储型"结构,使用双口BRAM将计算后的系数"对角交叉"变换位置后存储在原地址,利用双Bank存储模式满足蝴蝶变换单元的数据吞吐量;对Bar-rett约减算法进行修改,提出一种适用于多模值的模约减硬件单元,用加法和移位代替其中的乘法操作,减少乘法器资源的消耗.本方案使用Verilog HDL语言在Xilinx公司Artix-7系列XC7A200TFBG484-2型号FPGA芯片上进行了纯硬件优化实现.与当前研究相比,本设计在面积上减少12%-62.6%,时间上提升5.5%-49.8%.此外,我们还对高吞吐量的并行设计进行了优化实现,优化后的二并行、四并行架构时间上分别加速两倍、四倍,面积消耗仅为单结构的1.45倍、2.58倍.
传统的感知哈希算法通过提取图像特定属性生成感知哈希序列,难以充分利用原始图像全部特征信息,影响了基于感知哈希的图像内容认证与版权保护能力.本文提出一种基于双向生成对抗网络(Bidirectional Genera-tive Adversarial Network,BiGAN)的无监督感知哈希图像内容取证算法,基于编码网络、生成网络和判别网络间的双向迭代对抗,生成具有较强图像语义特征表示能力的感知哈希码;并通过在编码网络和生成网络间添加跳接层网络结构,将原始图像不同维度的特征信息传递到生成网络,提高生成网络语义特征学习能力与网络收敛速度;同时,在对抗损失中添加MSE误差损失,增强生成图像的视觉质量与细节表示能力;最后,基于网络间的多重迭代与对抗训练,输出兼具相同内容图像认证鲁棒性和不同内容图像区分敏感性的高性能图像感知哈希码.本研究首次采用大型图像数据库进行算法性能评价,实验结果表明基于双向生成对抗网络的感知哈希图像内容取证算法与当前其他优秀研究方案相比具有更强的图像内容取证性能.
鲲鹏920 CPU是2021年面世、全球第一款基于7纳米制造工艺的ARM 64位CPU,该CPU内置一个名为KAEzip的硬件加速引擎,其核心是一个硬件压缩算法,能通过硬件提升压缩与解压缩性能.相关研究表明,压缩算法的硬化与传统软件压缩算法相比具备明显性能优势.但大数据领域中的基础性系统软件都无法识别和使用这类算法.因此研究评估硬件压缩算法在大数据环境下的性能,发现揭示制约这类算法性能的关键因素以及可能存在的缺陷具有重要意义.为此,本文首先提出一种基于"生产-消费"模型的Spark任务性能模型,形式化地表示多维资源、压缩算法和Spark任务性能之间的内在关系,从理论上分析揭示出Spark下影响压缩算法性能的关键因素.然后提出一种三层架构支持Spark识别使用硬件压缩算法.这种分层架构为进一步调优硬件压缩算法在Spark中的性能提供了灵活性,也能复用到其他大数据系统软件.在此基础上本文以KAEzip 为实验对象,使用经典Spark 基准测试程序全面评估它在Spark中的性能,结合性能模型分析挖掘制约KAEzip性能的关键因素与根源.对KAEzip的测试表明:(1)硬件压缩算法可有效提升Spark性能.比如,KAEzip比snappy有最多13.8%的压缩性能优势、最多7%的解压优势和最多5.7%的实际应用场景下的性能优势;(2)磁盘的数据传输率与硬件压缩算法性能之间的不匹配是制约硬件压缩算法性能的重要因素;(3)压缩算法在Spark中的运行机制更易导致CPU的数据处理能力与硬件压缩算法性能不匹配,也制约着硬件压缩算法的性能.测试结果也表明KAEzip 在压缩小数据时会导致数据膨胀问题.为此,本文扩展三层架构分析揭示出导致该问题的根源,并结合压缩算法在Spark中的运行机制提出一种优化方法.硬件压缩算法作为压缩算法领域的新研究方向,本文的研究工作不仅可广泛用于优化内置于CPU中的硬件压缩算法在Spark下的性能,也有助于持续演化完善KAEzip和鲲鹏920 CPU.
近年来多模态情绪识别获得广泛关注,模态间的特征融合决定了情绪识别的效果,现有基于图的情绪特征融合方法多基于二元关系图,在处理三种及以上模态数据时难以实现有效的模态间特征融合,限制了多模态情绪识别的效果.为解决该问题,本文提出基于超图的多模态情绪识别模型(Multi-modal Emotion Recognition Based on Hypergraph,MORAH),引入超图来建立多模态的多元关系,以此替代现有图结构采用的多个二元关系,实现更加充分、高效的多模态特征融合.具体来说,该模型将多模态特征融合分为两个阶段:超边构建阶段和超图学习阶段.在超边构建阶段,通过胶囊网络实现对序列中每个时间步的信息聚合,并建立单模态的图,然后使用图卷积进行第二次信息聚合,并以此作为下一阶段建立超图的基础,得益于图胶囊聚合方法的加入,MORAH可以同时处理对齐数据和未对齐数据,无需手动对齐;在超图学习阶段,模型建立同一样本不同模态节点之间的关联,以及同类样本所有模态之间的关联,同时,在超图卷积过程中,使用分层多级超边来避免过于平滑的节点嵌入,并使用简化的超图卷积方法来融合模型之间的高级特征,以确保所有节点特征仅在必要时更新.在两个基准数据集上的综合实验表明,本文模型利用超图实现了对多模态数据之间多元关系的充分利用.与现有的先进方法相比,在CMU-MOSI数据集的未对齐数据上,MORAH将二分类准确率提高了1.3%,F1得分提高了1.1%.在CMU-MOSEI数据集的未对齐数据上,MORAH将二分类准确率和F1分数分别提高了0.2%.
深度学习方法促使多模态虚假新闻检测领域快速发展，现有的检测模型通常从全局角度学习新闻图文间的跨模态语义关联，并利用共享语义内容获取检测的关键信息.然而，新闻内部的局部语义差异可能会限制模型有效利用跨模态语义关联的能力，其中潜在的非共享语义内容作为重要线索能够有效揭示虚假新闻的篡改意图和目的.为了解决上述问题，本文提出了一种双分支线索深度感知与自适应协同优化的多模态虚假新闻检测模型.该模型首先从图像显著区域和文本语义单词中提取细粒度的新闻特征，并使用跨模态加权残差网络从中学习共享语义线索.同时，根据所有图像区域和文本单词之间的语义相关性，双分支图文线索感知模块显式地建模共享与非共享语义内容的语义关联.其中，线索关联优化分支对两类语义内容的关联边界持续迭代优化，促使模型准确区分非共享语义线索;线索关联分析分支刻画两类语义内容的可信程度，并在此基础上引导模型实现线索的自主融合.通过上述自适应协同优化框架，本文提出的模型能够在复杂新闻语境下进行线索的深度感知与融合，实现更准确、更可解释的多模态虚假新闻检测.在广泛使用的中英文真实数据集上的实验结果表明，本文提出的模型明显优于基线方法，在准确率和虚假新闻检测精确率上分别平均提高了4.85%和4.50%.
神经结构搜索技术可以在大量候选网络集合中搜索到适用于特定任务的神经网络结构.目前,大多数结构搜索网络的部署是针对英伟达GPU、英特尔CPU或谷歌TPU等硬件设备的.然而,将搜索到的架构迁移到一些AI专用加速器中,如寒武纪加速卡或华为Atlas推理加速器,推理效果却表现不佳.主要存在两方面的问题:在搜索空间设计层面,由于硬件架构设计对不同算子的支持存在差异,复用传统的搜索空间到专用神经网络加速器上,其推理效率不是最优的;在结构搜索层面,由于专用神经网络加速器在并行计算资源和数据流水通道等设计的不同,仅采用参数量、计算量作为搜索目标不能准确度量推理延迟,并且限制了神经结构搜索在精度和延迟上的探索空间.为了解决上述问题,本文提出一种基于硬件感知的多目标神经结构搜索方法,首先通过测试不同类型的卷积算子在目标硬件上的性能表现,使用非支配排序设计出定制化的高效搜索空间.然后,将延迟纳入搜索目标,提出一种启发式的混合粒度交叉算子,通过粗粒度阶段间交叉和细粒度阶段内交叉提高种群在多目标下的收敛性和多样性,更好地权衡神经网络的精度和推理延迟.本文主要针对国产寒武纪加速卡MLU270-F4进行了实验分析与方法验证,在CIFAR-10上搜索得到的MLUNet-S4精度比DARTS高0.14%的同时推理速度提升了4.7倍,相比于NSGANet精度仅下降0.04%的同时速度提升了5.5倍;在ILSVRC2012数据集上MLUNet-C相较于具有相同推理速度的Mobile Net V2和Mnas Net速度上提升了1.2倍的同时预测精度也分别提升了2.3%和0.2%,效果提升显著.
