Graph databases like Neo4j are powerful for analysing interconnected data, but performing advanced analyses such as spectral clustering often requires exporting data to external tools, incurring overhead and breaking the in-database workflow. While Neo4j offers various analytical capabilities, a dedicated, comprehensive plug-in for in-database spectral clustering has been a notable gap. To address this, we introduce SimKit1, a Java-based plug-in for Neo4j that enables true in-database spectral clustering by providing a suite of Cypher-callable procedures for end-to-end spectral clustering, including similarity graph construction (fully connected, epsilon-neighbourhood, k-nearest neighbourhood, and mutual k-nearest neighbourhood), Laplacian eigendecomposition of graph Laplacians, subsequent K-Means clustering of the resultant eigenvectors, and evaluation using silhouette coefficient and the Adjusted Rand Index. We evaluated SimKit on a range of feature-based datasets (Iris, Madelon, 20 Newsgroups) and native graph datasets (CORA, PubMed, CiteSeer). The results demonstrate that SimKit facilitates practical, end-to-end clustering workflows entirely within the Neo4j database system, achieving clustering quality comparable to established Python implementations.
With the widespread adoption of cloud computing and containerization technologies, large-scale clusters now host increasingly complex applications, yet the fragmentation of multidimensional resources (CPU, GPU, memory) has significantly degraded resource utilization efficiency. To address this challenge, this study investigates scheduling strategies for multidimensional resource fragmentation optimization. First, a unified quantitative metric is established by extending a task-statistics-based fragmentation measurement approach to threedimensional resource constraints (CPU, memory, GPU), enabling objective assessment of fragmentation levels. Subsequently, a multidimensional resource fragmentation optimization scheduling strategy is proposed based on the FGD framework. This approach employs TOPSIS to holistically address multidimensional resource fragmentation optimization. Building upon this methodology, we design and implement both two-dimensional and three-dimensional scheduling strategies. Comprehensive evaluation using an extended Kubernetes scheduler simulator with Alibaba production dataset reveals that: 1) All TOPSIS-based multi-dimensional strategies achieve performance comparable to or exceeding baseline approaches in their target dimensions while demonstrating synergistic effects.; 2) Two-dimensional strategies achieve optimal performance in their target dimensions but may degrade others 3) The three-dimensional optimization strategy TOPSIS-FGD-CGM can overcome the limitations of two-dimensional optimization strategies and demonstrates robust comprehensive performance.
Three-dimensional (3D) reconstruction based on aerial images of unmanned aerial vehicles (UAVs) is important for surveying and monitoring natural resources, supporting land cover analysis and geological hazard mapping. However, the performance of a fixed UAV docking station and manual deployments in large areas is limited by slow data acquisition and transmission. To accelerate data acquisition, emerging mobile UAV docking stations can collaboratively capture images in wide areas. Moreover, by leveraging edge computing power on UAVs and docking stations, the computations of 3D reconstruction can be performed locally, eliminating the need for data transmission. This paper builds a mathematical model for the entire process. The model can be decomposed into three associated problems: area partitioning, computation offloading, and mobile docking station path planning. The optimization objective is to minimize the execution time under the constraint of UAV battery power. A suboptimal solution is first derived using an enumeration-genetic algorithm and then fine-tuned using deep reinforcement learning. The experimental results validate the feasibility of using mobile docking stations for 3D reconstruction. In addition, the numerical results indicate that our proposed solution reduces execution time compared to the benchmark solution.
The growing disparity between computing speed and data access times poses a significant challenge to the management of data storage in large-scale supercomputers. To address these challenges, storage systems have evolved into hierarchical architectures with multiple levels that accommodate various hardware technologies. Each level offers a unique combination of performance, cost, and capacity. In this work, we introduce a paradigm shift in data management across the storage tiers by transitioning from block-level to file-level granularity to optimize data placement. Our online prediction model uses file characterization (input, log, checkpoint, work, and output) to predict file reuse with 97% accuracy, categorizing each file during usage and enabling proactive data management strategies. Using three real applications and a benchmark, we validate our work and demonstrate significant improvements in data hit rates, reaching 55% and 57% for LQCD and the benchmark applications, respectively. Even with lower performance improvements, our approach remains competitive with NEMO and NAMD, achieving a 3% gain over both LRU and LFU on NEMO and matching LRU performance while being 1% lower than LFU on NAMD.
