Parse Graph-Based Visual-Language Interaction for Human Pose Estimation

This study aims to improve human pose estimation under occlusion by hierarchically fusing vision and language in a parse graph framework. Language offers rich priors, such as spatial relations, but existing visual-language fusion using global features often weakens responses in occluded regions, causing alignment and localization errors. To address this, we propose Parse Graph-based Visual-Language interaction (PGVL) with a core novel Guided Module (GM), where low-level nodes preserve local features and high-level nodes preserve global features. PGVL performs hierarchical top-down decomposition and bottom-up composition via recursive cross-attention, guided by GM. GM enables high-semantic nodes to guide feature updates of cross-attention-processed low-semantic nodes, ensuring correct cross-modal fusion. We also design network based on PGVL, which achieves 68.2 MAP on CrowdPose (+0.7 over HRNet-W32), 82.1 MAP on AP-10K (+4.3 over CLAMP) and 79.3 MAP on Animal-Pose (+5.0 over CLAMP). PGVL and our network is validated on major pose estimation datasets. The code link is at https://github.com/lushbng/PGVL.