• 学术搜索
  • 科研智能体
    • Research Labs
    • AI 阅读
    • AI 文库
    • 深度研究
    • 学者亮点
  • 学术资源
    • AI2000
    • 期刊/会议
    • 学者库
    • 学术API
      MCP&Skill
    • 溯源树
    • 数据集
  • 知识沉淀
    • 学术空间
订阅小程序
旧版功能
aminer vip
开通会员低至0.73元/天
一次搞定AI科研
立即登录
  • English
  • 联系方式
    Journal of Machine Learning Research

    Journal of Machine Learning Research

    JournalISSN 1532-4435eISSN 1533-7928CCF 人工智能 A中科院 计算机科学 4区

    年发文量

    研究主题

    论文(10870)

    排序
    1NewKernel Mean Embedding Deviation Subspace for Unsupervised Learning with Heterogeneous Data
    Luoyao Yu,Lixing Zhu, Ruoqing Zhu,Xuehu Zhu

    This paper proposes a method for dimension reduction that preserves information in unsupervised learning with high-dimensional heterogeneous data, specifically targeting change point detection and clustering analysis. Our main strategy is to apply a Corrected Kernel Principal Component Analysis (CKPCA) method to construct the so-called kernel mean embedding deviation subspace. The approach efficiently identifies distributional changes in these dimension reduction subspaces for unsupervised dimension reduction. For change point detection, we demonstrate that the locations and number of change points in the dimension-reduced subspaces are identical to those in the original data. Furthermore, we extend this approach to clustering by embedding the original data into nonlinear lowerdimensional spaces, providing enhanced capabilities for clustering analysis. Additionally, we explain the necessity of using CKPCA, as the classical KPCA fails to identify the kernel mean embedding deviation subspace in these problems. Numerical studies on synthetic and real data sets suggest that the dimension reduction versions of existing methods for change point detection and clustering significantly improve the performance of current approaches in finite sample scenarios.

    2026
    引用
    AI阅读
    加入学术空间
    2NewWhy "classic" Transformers Are Shallow and A Depth-Enabling Technique
    Yueyao Yu, Yin Zhang

    Since its introduction in 2017, the Transformer has emerged as the leading neural network architecture, catalyzing revolutionary advancements in many AI disciplines. The key innovation in Transformer is a Self-Attention (SA) mechanism designed to capture contextual information. However, stacking up more layers of the same design has failed to produce trainable deeper Transformers. Thus far, various architectural modifications to the original design have been proposed to enable deeper depths for Transformer models, but a thorough understanding of this depth issue remains lacking. In this paper, we conduct a comprehensive investigation to substantiate the claim that the depth problem is caused by a phenomenon called token similarity escalation; that is, tokens grow increasingly alike after repeated applications of the SA mechanism. Our analysis reveals that, driven by the invariant leading eigenspace and large spectral gaps of attention matrices, token similarity provably escalates at a linear rate as the depth increases. This insight suggests a simple technique that surgically removes excessive token similarity without reducing the overall role of the SA mechanism, as is done by existing approaches. We perform a set of proofof-concept, small-scale experiments to show the viability of the proposed depth-enabling technique.

    2026
    引用
    AI阅读
    加入学术空间
    3NewTransfer Learning Via Regularized Random-effects Linear Discriminant Analysis
    Hongzhe Zhang,Arnab Auddy, Hongzhe Li

    Linear discriminant analysis is a widely used method for classification. However, the high dimensionality of predictors combined with small sample sizes often results in large classification errors. To address this challenge, it is crucial to leverage data from related source models to enhance the classification performance of a target model. This paper proposes a transfer learning approach via regularized random-effects linear discriminant analysis, where the discriminant direction is estimated as a weighted combination of ridge estimates obtained from both the target and source models. Multiple strategies for determining these weights are introduced and evaluated, including one that minimizes the estimation risk of the discriminant vector and another that minimizes the classification error. Utilizing results from random matrix theory, we explicitly derive the asymptotic values of these weights and the associated classification error rates in the high-dimensional setting, where the aspect ratio gamma := p/n as p, n -> infinity, with p representing the predictor dimension and n the sample size. Extensive numerical studies, including simulations, the analysis of the proteomics-based cardiovascular disease risk classification and the lipid traits classification problem with genotype data, demonstrate the effectiveness of the proposed approach.

    2026
    引用
    AI阅读
    加入学术空间
    4NewBridging Domain Invariance and Diversity: A Fine-Grained Risk Bound for Domain Generalization
    Xi Wang,Liang Bai,Xian Yang,Richard Yi Da Xu,Jiye Liang

    Domain-invariant representation learning and domain augmentation algorithms are two principal methodological paradigms for addressing domain generalization. They are widely employed in the machine learning literature to enhance domain invariance and domain diversity, respectively. However, existing risk bounds for domain generalization do not simultaneously capture the contributions of both approaches. This limitation arises because bounds derived directly in the original latent space are typically too coarse-grained and ambiguous to characterize how invariance and diversity jointly influence generalization. Since these two properties are often regarded as being inherently contradictory, it becomes difficult to disentangle and rigorously characterize their individual effects. To address this issue, we first observe that the latent representation space can be decomposed into several distinct subspaces, each exhibiting different characteristics and therefore being better suited for analyzing the respective roles of domain invariance and domain diversity. Building on this observation, we propose a unified analytical framework for domain generalization. Specifically, we introduce a Tri-Space Latent Representation and establish its unique decomposability via a direct-sum decomposition. Under this decomposition, each data representation can be uniquely partitioned into three components: domain-invariant features, spurious invariant features, and domain-variant features. Within this framework, we derive a finer-grained bound on the target-domain risk, which consists of two principal terms corresponding to domain diversity and invariant factors. By theoretically analyzing these two terms, we show that domain-invariant representation learning and domain augmentation are both effective and, crucially, compatible strategies for addressing domain generalization. Finally, we design two sets of experiments to empirically validate the relationship between domain invariance and domain diversity, and to examine their respective effects on domain generalization performance.

    2026
    引用
    AI阅读
    加入学术空间
    5NewDoubly Debiased Robust Subsampling for Transfer Learning
    Tao Wang,Weng Kee Wong

    This paper develops a general framework for doubly debiased robust subsampling for transfer learning. The setting arises when massive source datasets are computationally infeasible to use in full, while naive or heuristic subsampling leads to biased estimators that further inherit transfer bias under source-target distributional shifts. We resolve these challenges through two complementary debiasing mechanisms. Inverse probability weighting removes subsampling bias by ensuring that subsample-based estimators represent the full source distribution, while a target-based one-step refinement recenters estimators towards the target distribution, thereby mitigating transfer bias. These corrections are embedded within a distributionally robust optimization design that simultaneously controls worst-case target risk and enforces source-target alignment through maximum mean discrepancy. To optimize subsampling distributions, we propose a scalarized particle swarm algorithm that efficiently explores the robustness-alignment frontier by adjusting a single tuning parameter. We oracle inequalities, and minimax optimality under distributional uncertainty. Simulation studies and empirical applications in text sentiment and image recognition demonstrate that the proposed method consistently improves prediction accuracy and robustness compared with uniform subsampling, target-only training, and alignment-only approaches, and that both debiasing mechanisms are essential for reliable transfer.

    2026
    引用
    AI阅读
    加入学术空间
    立即登录,查看全部 10870 篇论文

    高被引作者

    作者引用发文
    Alexandre Gramfort1498937
    Alexandre Passos1489042
    Mathieu Blondel1488794
    Fabian Pedregosa1488213
    Jake VanderPlas1487942
    Olivier Grisel1486811
    Edouard Duchesnay1486811
    Vincent Dubourg1486811
    Vincent Michel1486811
    Peter Prettenhofer1486811

    高产作者

    作者引用发文
    Lei Chen537754
    Xuemin Lin491054
    Christian S. Jensen552452
    Michael I. Jordan885647
    Divesh Srivastava685046
    Samuel Madden626042
    Jeffrey Xu Yu482341
    Guoliang Li392839
    Beng Chin Ooi371134
    Lu Qin276233

    相关期刊

    Data Mining and Knowledge Discovery
    Machine Learning
    Conference on Uncertainty in Artificial Intelligence
    ACM SIGKDD Conference on Knowledge Discovery and Data Mining
    International Conference on Artificial Intelligence and Statistics
    International Conference on Learning Representations
    International Conference on Machine Learning
    Conference on Neural Information Processing Systems
    Advances in neural information processing systems
    AAAI Conference on Artificial Intelligence