Mixture-of-Experts (MoE) architectures enable efficient scaling of large language models by activating only a subset of parameters per input. However, existing MoE models suffer from two critical limitations: (1) inefficient token-to-expert routing that causes excessive communication overhead, and (2) expert homogenization that leads to redundant computations. Current approaches address these challenges separately, failing to achieve simultaneous improvements in both training efficiency and model performance. We present Expert-Token Resonance (ETR), a theoretically-grounded bidirectional routing mechanism that fundamentally reimagines expert-token interactions in MoE architectures. Our key insight is that optimal routing requires adaptive coordination between token-choice routing (TCR) during early training phases and expert-choice routing (ECR) in later stages. We prove that this dynamic approach maximizes training success rate (the probability of correct token-expert assignments) while reducing the expert capacity lower bound by up to 40

Graph Partitioning is widely used in many real-world applications such as fraud detection and social network analysis, in order to enable the distributed graph computing on large graphs. However, existing works fail to balance the computation cost and communication cost on machines with different power (including computing capability, network bandwidth and memory size), as they only consider replication factor and neglect the difference of machines in realistic data centers. In this paper, we propose a general graph partitioning algorithm WindGP, which can support fast and high-quality edge partitioning on heterogeneous machines. WindGP designs novel preprocessing techniques to simplify the metric and balance the computation cost according to the characteristics of graphs and machines. Also, best-first search is proposed instead of BFS and DFS, in order to generate clusters with high cohesion. Furthermore, WindGP adaptively tunes the partition results by sophisticated local search methods. Extensive experiments show that WindGP outperforms all state-of-the-art partition methods by 1.35 - 27 times on both dense and sparse distributed graph algorithms, and has good scalability with graph size and machine number.

The Mixtures-of-Experts (MoE) model is a widespread distributed and integrated learning method for large language models (LLM), which is favored due to its ability to sparsify and expand models efficiently. However, the performance of MoE is limited by load imbalance and high latency of All-to-All communication, along with relatively redundant computation owing to large expert capacity. Load imbalance may result from existing routing policies that consistently tend to select certain experts. The frequent inter-node communication in the All-to-All procedure also significantly prolongs the training time. To alleviate the above performance problems, we propose a novel routing strategy that combines load balance and locality by converting partial inter-node communication to that of intra-node. Notably, we elucidate that there is a minimum threshold for expert capacity, calculated through the maximal angular deviation between the gating weights of the experts and the assigned tokens. We port these modifications on the PanGu-Σ model based on the MindSpore framework with multi-level routing and conduct experiments on Ascend clusters. The experiment results demonstrate that the proposed LocMoE reduces training time per epoch by 12.68% to 22.24% compared to classical routers, such as hash router and switch router, without impacting the model accuracy.

Geo-location, also known as measurement report (MR) location, is a technique to determine the geographic location of user equipment (UE) and the behaviour attribute of telephone traffic based on wireless signals measured by the mobile communication network. The geographic location information can help to support network performance monitoring and evaluation. Considering accuracy and cost, we mainly adopt a hybrid location scheme combined with feature matching location and Weighted Centroid Correction Location (WCCL). As for feature matching location, over 20 billion samples gathered from tens of thousands of cells daily updated. Due to the vast data scale, feature analysis encounters a severe performance bottleneck. To address this problem, we design the indexed parallel decision tree (indexPDT) operator and integrate it into WindTensor, a self-innovated distributed machine learning (ML) engine. indexPDT is a classifier unit of the random forest (RF) algorithm with a novel cache structure. It performs structured cache processing on the dataset's meta-information, which can accompany the splitting of nodes. The cache structure can be quickly converted into statistical information to help find the optimal splitting point, effectively reducing memory usage and improving performance. Under the public datasets testing on 5 nodes, the mean speedup ratios are 86x and 3x compared with SparkML and XGBoost, respectively. In the Geo-location scenario, for a single cell, the speedup ratios are 82x and 4x compared with SparkML and XG Boost, respectively.

Fuzzy path query is widely used to find the deep association of entities in many real-world applications such as knowledge graph answering and social network analysis. However, existing engines fail to support fuzzy path queries on large property graphs due to the imprecise string matching and indefinite search space. In this paper, we propose an extremely fast graph query engine KBQA, which can perform semantic matching in both entities and properties, and search arbitrarily long paths efficiently. Facing the performance problem, KBQA designs two-phase filtering strategy to accelerate candidate selection. Also, bitwise operations are adopted for fast graph exploration. Furthermore, KBQA adaptively prunes unpromising search paths based on path similarity. Extensive experiments show that KBQA outperforms all state-of-the-art graph databases by 2 × ∼ 10 × and searches all 6-hop paths within ten seconds. Our system has been applied in the ICT field and has achieved remarkable results.

Graph partitioning (GP), a.k.a. community detection, is a classic problem that divides the node set of a graph into densely-connected blocks. Following prior work on the IEEE HPEC Graph Challenge benchmark and recent advances in graph machine learning, we propose a novel RAndom FasT Graph Partitioning (RaftGP) method based on an efficient graph embedding scheme. It uses the Gaussian random projection to extract community-preserving features from classic GP objectives. These features are fed into a graph neural network (GNN) to derive low-dimensional node embeddings. Surprisingly, our experiments demonstrate that a randomly initialized GNN even without training is enough for RaftGP to derive informative community-preserving embeddings and support high-quality GP. To enable the derived embeddings to tackle GP, we introduce a hierarchical model selection algorithm that simultaneously determines the number of blocks and the corresponding GP result. We evaluate RaftGP on the Graph Challenge benchmark and compare the performance with five baselines, where our method can achieve a better trade-off between quality and efficiency. In particular, compared to the baseline algorithm [1] of the IEEE HPEC Graph Challenge, our method is 6.68x - 23.9x faster on graphs with 1E3 - 5E4 nodes and at least 64.5x faster on larger (IE5 node) graphs on which the baseline takes more than 1E4 seconds. Our method achieves better accuracy on all test cases. We also develop a new graph generator to address some limitations of the original generator in the benchmark.

Graph algorithms (e.g., k-hop queries) are widely used to find the deep association of data in various real-world applications such as business recommendation and fraud detection. However, most of the data are still stored in relational database (i.e., RDBMS) and the performance is rather limited when processing graph queries on RDBMS due to the inherent hardness of complicated table join. In this paper, we propose a fast interactive engine SQLG+, which can be integrated to any RDBMS and enable them to process k-hop graph queries efficiently. Different from naive table-join implementations, SQLG+ caches important nodes with their adjacency lists in memory (i.e., graph cache) and generates a hybrid query plan which combines the ability of graph cache and RDBMS. Also, SQLG+ removes duplicates in the end of each hop (using AdaptiveSet) and expands the frontiers in different ways. Furthermore, dynamic BFS/DFS switch is adopted to achieve the balance between query performance and memory occupation. Extensive experiments show that SQLG+ outperforms the state-of-the-art RDBMS-based implementations by up to several orders of magnitude and is even comparable to the fastest graph databases.

Graph algorithms (e.g., triangle counting) are widely used to find the deep association of data in various real-world applications such as friend recommendation and junk mail detection. However, even if using the massive parallelism of GPU, existing methods fail to run triangle counting queries efficiently on various large graphs. In this paper, we propose a fast hybrid algorithm HTC, which can utilize both vertex-parallel and edge-parallel paradigm and deliver much better performance on GPU. Different from current GPU implementations, HTC adaptively selects different parallel paradigm for different vertices. Also, bitwise-based intersection on segmented bitmap is proposed instead of naive binary search. Furthermore, preprocessing techniques like graph reordering and recursive clipping are adopted to optimize the graph structure. Extensive experiments show that HTC outperforms all state-of-the-art triangle counting implementations on GPU by 1.2x~42x.