TSFD-Net: Tissue specific feature distillation network for nuclei segmentation and classification

Nuclei segmentation and classification of hematoxylin and eosin-stained histology images is a challenging task due to a variety of issues, such as color inconsistency that results from the non-uniform manual staining operations, clustering of nuclei, and blurry and overlapping nuclei boundaries. Existing approaches involve segmenting nuclei by drawing their polygon representations or by measuring the distances between nuclei centroids. In contrast, we leverage the fact that morphological features (appearance, shape, and texture) of nuclei in a tissue vary greatly depending upon the tissue type. We exploit this information by extracting tissue specific (TS) features from raw histopathology images using the proposed tissue specific feature distillation (TSFD) backbone. The bi-directional feature pyramid network (BiFPN) within TSFD-Net generates a robust hierarchical feature pyramid utilizing TS features where the interlinked decoders jointly optimize and fuse these features to generate final predictions. We also propose a novel combinational loss function for joint optimization and faster convergence of our proposed network. Extensive ablation studies are performed to validate the effectiveness of each component of TSFD-Net. The proposed network outperforms state-of-the-art networks such as StarDist, Micro-Net, Mask-RCNN, Hover-Net, and CPP-Net on the PanNuke dataset, which contains 19 different tissue types and 5 clinically important tumor classes, achieving 50.4% and 63.77% mean and binary panoptic quality, respectively. The code is available at: https://github.com/Mr-TalhaIlyas/TSFD.