Multi-agent shape models for hip landmark detection in MR scans

Landmark detection is an essential step in the diagnosis of bone pathologies and pelvis morphometry. However, human abilities to quantify pelvic abnormalities on early asymptomatic stages are limited. Hence, we propose a Deep Learning based method for automatic landmark detection on multi-modality hips magnetic resonance (MR) scans. Our method is based on a synergistic analysis of appearance and shape information by using deep networks for the detection of landmark candidate locations and then adjusting these locations using interlandmark spatial properties. The algorithm's backbone predicts initial landmark candidate points using Feature Pyramid Networks (FPN) and Region Proposal Networks (RPN), the candidate points are then fine-tuned using a multi-agent collaborative Reinforcement Learning module based on the patch level information and the current positioning of the other landmarks. Our model was trained on 140 pelvic MR scans that were manually labelled by an expert physician using 16 landmarks that defined 6 clinical measurements and evaluated on 36 samples. Our two steps approach provides faster and more accurate performances when compared to other appearance only state-of-the-art landmark detection approaches based on UNET or FPN+RPN. Our best model which is based on a UNET backbone model with a (TD3 + wide and deep neural network) fine tuner gives an average of 1.74 mm over all the landmarks, where 67% of the proposed landmarks are within the spatial matching error of at most 2mm.