Cooperative multi-agent reinforcement learning under partial observability requires decentralized agents to act from incomplete local information. Value-decomposition methods such as VDN and QMIX do not control when teammate-related knowledge may influence an agent’s local utility, and an uncontrolled teammate-conditioned term can distort decisions at timesteps where no ally is observable. We propose CA-QMIX, a context-aware value factorization method that uses ally visibility as a knowledge signal for decentralized agent control. Each agent combines an individual decision branch with an attention-based cooperative knowledge branch through a visibility-conditioned gate that scales the cooperative contribution and is hard-masked to zero when no ally is observable. The gated utilities are combined by a QMIX-style monotonic mixer, which we prove preserves Individual-Global-Max consistency, and a mixer-level Δ-Q regularizer discourages cooperative corrections that lower the greedy value of the base branch. Experiments on seven SMAC v1 and SMAC v2 scenarios against six value-decomposition baselines show that CA-QMIX attains the highest mean final win rate and the best mean rank over the seven scenarios and improves on QMIX in all three randomized SMAC v2 scenarios. Ablations show that most of the gain comes from the cooperative branch, that its visibility conditioning helps when correctly aligned (misaligned visibility degrades performance under randomized placement), and that the form of the gate is not critical: a binary gate suffices and the hard no-ally mask has no measurable cost.
Offline reinforcement learning aims to learn effective policies from fixed datasets without online interaction, necessitating conservative constraints to mitigate the out-of-distribution (OOD) issue. Although existing approaches alleviate this issue through conservative constraints or policy regularization, they still struggle to effectively exploit high-quality samples in offline datasets. To address these issues, this study proposes Dual Advantage-Guided Offline Reinforcement Learning (DAG). The approach designs an Advantage-Guided Variational Autoencoder (AG-VAE) to reconstruct the behavior policy from offline datasets. The policy learns from both reconstructed actions and high-advantage dataset actions, with advantage weighting facilitating policy generalization. To evaluate the effectiveness of the proposed approach, DAG is assessed on MuJoCo robotic control tasks and AntMaze tasks from the D4RL benchmark across datasets with varying data quality. Experimental results demonstrate that DAG effectively exploits advantage information and outperforms existing offline reinforcement learning approaches across multiple tasks. Compared with the TD3+BC baseline, DAG improves the total normalized score by 9.69% on MuJoCo and 10.06% on AntMaze.
Dynamic typing brings convenience to programming but also poses challenges for compilers to generate efficient code. We propose a profiling-based method that collects type information by warming up and running JavaScript programs. By monitoring the program execution and recording the type information of bytecode and variables, a rule-based type inference scheme and a supplementary solution supporting JavaScript source code are presented. With static program analysis and the JavaScript type system, constraints and inferences are applied to variable types to generate more accurate and reliable type information. This paper implements a type inference system that takes JavaScript source code as input and outputs type information files. Experiments on benchmarks such as SunSpider show that, compared with WebAssembly binaries compiled without type information, those with type information achieve an average performance improvement of 6.18% and a maximum performance improvement of 58.9% in execution efficiency.
Traditional cloud resource management software typically perform workload prediction, VM allocation and migration policies independently, yielding sub-optimal resource utilization and excessive migration overhead. This study presents PRIMA, an optimization-driven prediction-assisted resource management framework for efficient resource management in cloud environment. The framework leverages neural model to forecast memory and CPU utilization of VMs, wherein Raindrop Optimization Algorithm is employed to enhance prediction accuracy. Based on the predicted workload, VMs are proactively mapped to physical servers to optimize resource utilization and minimize server overload. Additionally, a multi-objective VM migration strategy consolidates workloads by minimizing active servers without compromising service quality. Experimental evaluation using real-world Bitbrains workload traces along with additional validation using the Google Cluster Dataset demonstrates that PRIMA attains consistent improvements in resource management performance, yielding up to 9.63% reduction in power consumption, 75% reduction in conflicting servers and 73.24% reduction in VM migrations, and a 13.46% enhancement in resource utilization in comparison to RNN-LSTM, SVM, ARIMA, and HHO-based prediction-assisted baseline frameworks. Overall, PRIMA provides a unified and proactive alternative to conventional prediction-assisted cloud resource management approaches, making it suitable for intelligent cloud computing systems.
Comparing stochastic transition systems with heterogeneous action interfaces requires a discrepancy that accounts for state differences, action semantics, and successor behavior. We formulate cross-system comparison as a two-player minmax imitation game and define a Bellman-Kantorovich operator combining state and action discrepancies with optimal transport. We establish the existence and uniqueness of the strategy-induced fixed point and characterize the minmax game value. For finite systems, we develop a sample-based learning algorithm and derive a high-probability finite-sample certificate for the output of the asynchronous estimator and an almost-sure convergence guarantee for asynchronous updates with increasing successor batches. For scalable finite-state computation, we use state-value iteration, duplicate-support aggregation, and bound-based action screening to reduce storage and exact transport solves. Experiments on shared-action and heterogeneous-action systems, warehouse navigation, and real robot transition data evaluate the discrepancy under transition perturbations, heterogeneous action interfaces, data shift, and finite-state computational scaling.
