Recent years have witnessed the growing deployment of optical circuit switches (OCS) in commercial GPU clusters (e.g., Google A3 GPU cluster) optimized for machine learning (ML) workloads. Such clusters adopt a three-tier leaf-spine-OCS topology, servers attach to leaf-layer electronic packet switches (EPSes); these leaf switches aggregate into spine-layer EPSes to form a Pod; and multiple Pods are interconnected via core-layer OCSes. Unlike EPSes, OCSes only support circuit-based paths between directly connected spine switches, potentially inducing a phenomenon termed routing polarization, which refers to the scenario where the bandwidth requirements between specific pairs of Pods are unevenly fulfilled through links among different spine switches. The resulting imbalance induces traffic contention and bottlenecks on specific leaf-to-spine links, ultimately reducing ML training throughput. To mitigate this issue, we introduce a leaf-centric paradigm to ensure traffic originating from the same leaf switch is evenly distributed across multiple spine switches with balanced loads. Through rigorous theoretical analysis, we establish a sufficient condition for avoiding routing polarization and propose a corresponding logical topology design algorithm with polynomial-time complexity. Large-scale simulations validate up to 19.27

Traditional knowledge graph completion (KGC) methods rely solely on structural information and struggle with sparsity, while Large Language Models (LLMs) address these limitations through rich world knowledge and strong context modeling. Fine-tuning LLMs is effective but costly, while non-fine-tuned LLMs are efficient but suboptimal. To address this trade-off, we propose FLAME, a framework that extracts context-aware hidden states from intermediate layers of frozen LLMs to train data-efficient KGC classifiers. We bridge LLM-KG semantic gaps via subgraph-based entity descriptions and employ sliced mutual information (SMI) to quantify task-relevant information in representations. Experiments demonstrate that FLAME achieves 47% improvement over non-fine-tuned LLM baselines and, to our knowledge, is the first to achieve fine-tuned performance with 188× memory efficiency and 26.11× speedup.

Scaling large language models by increasing parameters and training data is increasingly constrained by limited high-quality corpora and rising communication costs. This work explores an alternative axis: increasing per-token computation without expanding parameters, by internalizing latent Chain-of-Thought (CoT) into pretraining. We propose Pretraining with Token-Level Adaptive Latent CoT (adaptive latent CoT), where the model generates a variable-length latent CoT trajectory before emitting each token – allocating longer trajectories to difficult tokens and shorter (or even zero) trajectories to easy ones. Importantly, this behavior emerges naturally from one-stage pretraining on general text and reduces computation in both training and inference via token-wise adaptive halting. Experiments with Llama architectures show that adaptive latent CoT consistently improves language modeling perplexity and broad downstream accuracy, even with fewer training FLOPs than prior recurrent baselines.

Recent years have witnessed the growing deployment of optical circuit switches (OCSes) in commercial GPU clusters (e.g., Google’s A3 GPU cluster) optimized for machine learning (ML) workloads. Such clusters adopt a three-tier leaf–spine–OCS topology: servers attach to leaf-layer electronic packet switches (EPSes), these leaf switches aggregate into spine-layer EPSes to form a pod, and multiple pods are interconnected via core-layer OCSes. Unlike EPSes, OCSes only support circuit-based paths between directly connected spine switches, potentially inducing a phenomenon termed routing polarization, which refers to the scenario where the bandwidth requirements between specific pairs of pods are unevenly fulfilled through links among different spine switches. The resulting imbalance induces traffic contention and bottlenecks on specific leaf-to-spine links, ultimately reducing ML training throughput. To mitigate this issue, we introduce a leaf-centric paradigm to ensure traffic originating from the same leaf switch is evenly distributed across multiple spine switches with balanced loads. Through rigorous theoretical analysis, we establish a sufficient condition for avoiding routing polarization and propose a corresponding logical topology design algorithm with polynomial-time complexity. Evaluations on a 128-NPU testbed show up to 23.7% higher training throughput than state-of-the-art pod-centric approaches, while large-scale simulations validate up to 19.27% throughput improvement and a 99.16% reduction in logical topology computation overhead compared to mixed integer programming (MIP)-based methods.

From-Scratch Name Disambiguation (SND) groups papers sharing an ambiguous name into clusters of distinct real-world authors. Existing methods suffer from two critical limitations: (1) inherent long-tailed author distribution biases representation learning, causing over-merging of tail authors; (2) existing cluster number estimation methods are unreliable for long paper sequences, hindering large-scale deployment. We propose GRAND-HC, a complete end-to-end SND framework. We construct a heterogeneous paper graph via co-author, co-organization, and co-venue relations, using a graph attention network as the embedding backbone. Harmony Contrastive Learning (HCL) dynamically reweights training loss to suppress overfitting to prolific authors, learning discriminative embeddings. A Graph-Refined Distance Matrix (GRDM) leverages graph topology to optimize pairwise distances, further preventing tail author over-merging. Meanwhile, a lightweight Paper Compression Module (PCM) achieves accurate cluster number estimation across varying scales. Finally, Hierarchical Agglomerative Clustering outputs the final clusters. Extensive experiments demonstrate state-of-the-art macro F1 performance. GRAND-HC has been deployed in a billion-scale academic database. Source code: https://github.com/baokou-fw2/GRAND-HC.

Deep learning has demonstrated remarkable generalization capability with independent and identically distributed (i.i.d.) training and test data, however, it often struggles with data drawn from different, albeit causally related, distributions. This problem is generally known as Out-of-Distribution (OoD) generalization. While there is a plethora of algorithms proposed for OoD generalization, the current understanding of the data commonly employed to evaluate these algorithms remains relatively naive. In this study, we identify two distinct types of distribution shifts, namely diversity shift and correlation shift, that are ubiquitous in various OoD datasets. We propose a quantifiable formal definition for the two shifts and show that the performance of OoD algorithms is upper bounded by them. To validate our theoretical insight, we evaluate a number of OoD generalization algorithms across two groups of datasets from both classification and object detection areas, each dominated by one of the shifts, exposing the strengths of the algorithms against one shift as well as their limitations against the other. We further proved that all performance degradations according to data distribution shifts can be attributed to these two types of shifts defined in our paper. The benchmark integrates existing datasets and algorithms from different research areas that seem unrelated into a coherent picture, which may serve as a foundation for future OoD generalization research.

Robust aerial target detection for autonomous UAV-on-UAV pursuit is severely hindered by continuous scale drift, long-tailed scale imbalance, and flight-induced visual noise, rendering standard empirical risk minimization strategies poorly aligned with real-world deployment. To address these challenges, we propose a scale-aware robust optimization framework that performs group-wise minimax optimization over scale-partitioned data, ensuring balanced robustness across long-, mid-, and close-range engagement regimes. We further introduce an uncertainty-rectified regression loss to suppress noise-driven errors without discarding informative hard examples, complemented by a control-aligned center accuracy penalty that prioritizes the localization precision required for stable flight control. Extensive experiments demonstrate that our method yields substantially improved robustness under visual degradation, with significantly slower decay in detection mAP and center-point accuracy compared to baselines. Validated through both photorealistic simulations and real-world flight tests, our system achieves on an embedded NVIDIA Orin NX platform, confirming its practical efficacy for high-speed interception.

Self-attention selects information freely across the sequence, but across depth, Transformers merely add each layer's output to the residual stream, so later layers cannot selectively reuse earlier-layer representations. Recent cross-layer methods improve this flow but operate on hidden states outside attention, adding state beyond the key-value cache at inference–a cost that becomes increasingly salient as modern LLMs compress the cache with grouped-query and multi-head latent attention. We introduce Depth-Attention, which performs this selection inside the attention module itself: before a layer attends over the sequence, its query attends over the keys of earlier layers at the same token position and mixes their values into the value that self-attention then reads. Because Depth-Attention reuses the standard attention queries, keys, and value-cache slots, storing depth-mixed values in place of the original values, it adds no parameters and introduces no persistent inference state beyond the standard key-value cache–the same cache size as a vanilla decoder and less than hidden-state-based cross-layer methods. On Qwen3-style decoders at 1.5B and 3B parameters, Depth-Attention attains the lowest perplexity and the highest average downstream accuracy, improving over the vanilla Transformer by up to 2.3 accuracy points and surpassing strong cross-layer baselines in perplexity and average accuracy, while adding under 0.01

From-Scratch Name Disambiguation (SND), a core Author Name Disambiguation (AND) task, aims to group papers with identical ambiguous names into clusters of distinct real-world authors. However, existing SND methods suffer from two critical limitations: first, the inherent long-tailed uneven distribution, where most papers belong to a few prolific authors, severely biases representation learning, causing low-discriminative embeddings and over-merging of tail authors; second, cluster number estimation methods are unreliable and poorly scalable for long sequences, restricting real-world deployment. To address these issues, we propose GRAND-HC, an end-to-end SND framework with three components. We first construct a heterogeneous paper graph based on co-author, co-organization and co-venue relations, and adopt a graph attention network as the backbone. Then, harmony contrastive learning (HCL) dynamically reweights loss to suppress overfitting to prolific authors, learning highly discriminative embeddings. On this basis, a graph-refined distance matrix (GRDM) leverages graph topology to optimize pairwise distances, preventing over-merging of tail authors. Meanwhile, a lightweight Paper Compression Module (PCM) achieves accurate cluster number estimation across varying scales, eliminating the long-sequence modeling defect. Finally, Hierarchical Agglomerative Clustering outputs the final clusters with the optimized distance matrix and estimated cluster number. Extensive experiments demonstrate that GRAND-HC outperforms state-of-the-art models on macro F1 score. Furthermore, GRAND-HC has been deployed in a billion-scale academic database.

Visual token pruning methods effectively mitigate the quadratic computational growth caused by processing high-resolution images and video frames in vision-language models (VLMs). However, existing approaches rely on predefined pruning configurations without determining whether they achieve computation-performance optimality. In this work, we introduce , a novel framework that formulates visual token pruning as a Pareto configuration optimization problem to automatically identify optimal configurations. Our approach employs continuous relaxation and straight-through estimators to enable gradient-based search, solved via the Augmented Lagrangian method. Extensive experiments across 8 visual benchmarks demonstrate that effectively approximates the empirical Pareto frontier obtained through grid search and generalizes well across various pruning methods and VLM architectures. Furthermore, through learnable kernel functions, we investigate layer-wise pruning patterns and reveal that multi-step progressive pruning captures VLMs' hierarchical compression structure, achieving superior accuracy-efficiency trade-offs compared to single-layer approaches.

This paper tries to answer a question: “Can we achieve spatial-selective transmission on IoT devices?” A positive answer would enable more secure data transmission among IoT devices. The challenge, however, is how to manipulate signal propagation without relying on beamforming antenna arrays which are usually unavailable on low-end IoT devices. We give an affirmative answer by introducing SpotSound, a novel acoustic communication system that exploits the diversity of multi-path indoors as a natural beamformer. By judiciously controlling the way how the information is embedded into the signal, SpotSound can make the signal decodable only when the signal propagates along a certain multipath channel. Since the multipath channel decorrelates rapidly over the distance between receivers, SpotSound can ensure the signal is decodable only at the target position, achieving precise physical isolation. SpotSound is a purely software-based solution that can run on most IoT devices where speakers and microphones are widely used. We implement SpotSound on Raspberry Pi connected with COTS microphone and speaker. Experimental results show that SpotSound could precisely focus its signal on spots with customized sizes ranging from 0.04m2 to 0.5m2.

The remarkable success of Chain-of-Thought (CoT), which enhances performance by scaling generation steps at test-time, inspires us to ask: can we leverage a similar scaling of computational steps during pretraining to improve the generation of each individual token? To address this, we propose a novel pre-training methodology: Pretraining Language Models with Latent Thoughts (PonderLM-2). Our approach pretrains a language model (LM) to first generate an intermediate latent thought—the last hidden state of the current position—which is then used as input to predict the actual subsequent token. This additional computational step enables the LM to refine its prediction within unconstrained continuous space. Our experiments demonstrate that, at an identical inference cost, a LM that generates one additional latent thought per token outperforms a standard model with double the parameters. For instance, our PonderLM-2-Pythia-1.4B, pretrained on 300B tokens from the Pile, significantly surpasses the vanilla Pythia-2.8B trained on the same data on both language modeling and a range of general downstream tasks. Furthermore, increasing the number of latent thoughts generated before each actual token—forming a chain analogous to CoT—consistently improves the model's performance.

In real-world scenarios, training and test data are often collected in diverse settings, leading to domain shifts arising from evolving environments and selection bias. While causality-inspired methods have shown promising results in tackling the out-of-distribution (OOD) generalization issue, prior methods treat the discovered differences across domains as confounding variables. While effective in handling domain differences (i.e., unseen environmental features in test data), they may fail when confronted with intricate spurious correlations in real-world datasets. In this study, we first analyze this limitation to inadequate modeling of causal intervention and derive the OOD generalization bound to explain the challenges it introduces. To address this problem, we propose a modified causal intervention approach to mitigate various types of confounders. Motivated by the mathematical formulation of our modified causal intervention, we introduce the Causal Feature Selection Module (CFSM) to suppress model weights on both domain-differences features and spurious correlation features. Integrated within the Base Feature Extraction Module, In-Sample Module, and Cross-Sample Module (B-I-C architecture), CFSM collectively neutralizes the confounding effects arising from both domain discrepancies and correlation distinctions, thereby achieving causal feature selection. Under mild assumptions, we prove that the proposed CFSM method can achieve strictly lower OOD errors. Further experiments conducted on various benchmark datasets demonstrate the effectiveness of the proposed method. Compared to previous deconfounding methods, our method not only mitigates the effect of domain-differences features but also the hard-to-identify spurious correlation features, achieving significant improvements in two-dimensional OOD generalization.

This paper proposes a novel address mapping mechanism for multi-robot communication and collaboration systems, named SPing. SPing addresses each robot with a dynamic physical-world address - an encoding of the robot's physical location - rather than a pre-assigned digital-world ID (e.g., the IP address). This enables a "see-and-point" communication mode for robots: a robot can establish an immediate connection pointing to any other robot it intends to collaborate with in its visual field, without relying on a pre-existing multi-robot network. This on one hand improves the robustness and usefulness of multi-robot systems in uncertain and unstructured environments where network infrastructures are unavailable. On the other hand, it makes the robots' communication behavior tightly coupled with and more supportive of the collaboration tasks in the physical world. We build an end-to-end prototype of SPing and evaluate its performance in both static and mobile scenarios. The results show that SPing can always establish a connection precisely pointing to the target receiver with an average matching rate of 99.58%, and a spatial resolution of 0.3 m similar to 0.5 m.

In recent years, Large Language Models (LLMs) have achieved remarkable success across various domains, sparking interest in extending their capabilities to graph structured data. However, leveraging LLMs for graph data poses significant challenges due to the inherent mismatch between graph and text modalities. Existing approaches primarily rely on two strategies: Graph-to-Text Translation, which describes graph structures in natural language to enable LLMs to process structural information, and Graph-to-Token Conversion, which transforms graphs into sequences of tokens aligned with text tokens. Although these methods have achieved a certain degree of integration between LLMs and graph data, they still struggle to fully capture the complex structure in real-world graphs and fail to provide a global view of the long-range dependencies—the relationships or interactions between nodes that are far apart in the graph. To bridge this gap and enhance LLM’s understanding of the graph structure, we propose Curvature Enhanced Large Language Model (CELLM), a novel architecture that integrates advanced graph structural information with discrete graph curvature to offer a global and geometric perspective. Graph curvature leverages local curvature measurements to derive global insights, such as assessing overall connectivity, identifying bottlenecks, or detecting hubs. This more expressive structural measure can provide LLMs with enhanced capabilities to capture and comprehend complex graph structures. Additionally, we implement a task-specific tuning procedure to further improve the structure understanding within LLMs. Extensive experiments demonstrate the effectiveness of our proposed CELLM across graph-related tasks, highlighting its potential in improving the expressiveness and understanding of LLMs when applied to graph modalities.

Graph coarsening is a graph dimensionality reduction technique that aims to construct a smaller and more tractable graph while preserving the essential structural and semantic properties of the original graph. However, most existing methods rely on pair-wise similarity matching, where each node independently searches for its best partner based on global information. This selfishness matching paradigm incurs substantial computational and memory overhead. To address this problem, we shift to a non-selfishness principle that prioritizes the collective interference of neighborhood in coarsening, and propose an efficient method named NOPE, which achieves linear memory consumption and near-linear computational complexity in the number of nodes. Furthermore, we derive a faster variant NOPE*, which reduces O(δḋ) interference evaluation to O(d) based on the local isotropy assumption, and consequently alleviates the computational bottleneck for high-degree nodes. Experimental results show that NOPE* achieves 1.8-10×speedup over NOPE and surpass almost all baselines with 1-3 orders of magnitude acceleration. Meanwhile, learning on coarsened graphs yields comparable performance to original graphs, and can even show superior performance over LLM-based graph reasoning owing to compact graph information. The code can be available at https://github.com/dazonglian/NOPE-main.