Automated detection of HER 2 amplification status in FISH images using object localization networks

The HER2 gene amplification status is important for therapy assessment in breast and gastric cancer. It is routinely assessed visually by pathologists based on fluorescence in situ hybridization (FISH) imaging. To improve the accuracy and objectivity of this procedure, we have developed a two-step pipeline for the detection, localization and classification of interphase nuclei and fluorescence signals based on RetinaNet-based object detection networks. We demonstrate that the nucleus-level detection accuracy of this pipeline is on par with a team of pathologists and achieves a 96% accuracy for the classification of whole FISH slides. Moreover, by classifying each nucleus twice, our system provides rich medical reports increasing both robustness and interpretability. Our pipeline is a first step in automating the evaluation of the HER2 status of tumors using FISH images and in optimizing the documentation of each tumor sample by automatically annotating and reporting of the HER2 gene amplification status. For more details, refer to (Zakrzewski et al., 2019).