Instance-aware Multi-task Learning for Nuclei Segmentation

Nuclei segmentation is critical for computational pathology analysis. Most previous methods employ pixel-wise classification or regression for automatic nuclei segmentation, without describing nucleus instances as individual entities at the feature level. To address the above limitation, we propose an instance-aware multi-task learning framework that strengthens a pixel-wise prediction branch with an instance-wise prediction branch. The instance-wise prediction branch leverages learnable cell-level queries, enabling the model to capture positional information and visual representations for individual nuclei. Concretely, we introduce an instance-disentangling feature learning module that effectively aligns the embeddings of the object-level queries with pixel-wise decoder features from the first branch. Further, we design a dual-branch unified post-processing algorithm that aggregates the complementary outputs of both branches for computing the instance segmentation results. Experimental results demonstrate that our framework achieves competitive performance on a wide range of nuclei segmentation benchmarks. The code and model weights are released in https://github.com/lhaof/IML .