Dynamic Topological Graph Reinforcement Learning Framework for Solving Routing Optimization Problems

Recently, the Learning-to-Improve paradigm has shown significant promise in solving routing optimization problems. However, most studies have mainly focused on transformer-based encoder architectures, which tend to aggregate information across the entire instances. These methods often overlook the crucial topological relationships between nodes, leading to excessive computational overhead and suboptimal performance as the problem size grows. To tackle these limitations, we design a dynamic topological graph reinforcement learning (DTGRL) framework that utilizes a bi-level strategy to guide the agent toward the most valuable actions. At the encoding stage, we design a novel Dynamic Topological Graph Attention (DTGA) module. It integrates geometric priors and semantic similarity to generate adaptive structural topology masks that effectively filter out global noise while enforcing a locality-centric inductive bias. At the decoding stage, we introduce a Sparse Compatibility Layer (SCL) that prunes the expansive action space to encourage exploration while exploiting higher-value actions. In our experiments, we train the agent using an n-step proximal policy optimization (PPO) approach. The experimental results demonstrate that DTGRL significantly outperforms the existing Learning-to-Improve methods and showcases superior generalization capabilities across various datasets and benchmarks of different sizes.