Retinal multi-lesion segmentation by reinforcing single-lesion guidance with multi-view learning

The primary prerequisite for multi-lesion segmentation is the simultaneous detection of multiple lesions. Numerous techniques based on the frameworks of simultaneous multi-lesion segmentation and repetitive single-lesion segmentation are continually refining their models to meet this requirement. While simultaneous multi-lesion segmentation techniques may fully exploit interactions between distinct lesions, single-lesion segmentation methods can concentrate only on a particular lesion. However, since different lesions exhibit distinct patterns, it is challenging for these sophisticated models to work properly when several lesions are present simultaneously. We propose a retinal multi-lesion segmentation method by reinforcing single-lesion guidance with multi-view learning. To the best of our knowledge, this is the first work to formulate the retinal multi-lesion segmentation task as a multi-view task. In the multi-view method, each segmentation branch incorporates context guidance of the particular lesion at the input to focus more attention on specific lesions while producing multi-lesion outcomes. For implementing the multi-view method, we design a two-level hierarchical heterogeneous network, whose core element is the multi-view segmentation branches at the second level. The first level provides the context guidance as multi-view cues. Concretely, our network incorporates two complementary models (i.e., U-Net and TransU-Net). Compared with U-Net focusing on local receptive fields, TransU-Net augments U-Net by Transformer, which is adept at establishing long-distance dependence through global attention. The entire network consists of TransU-Net at the first level and a subsequent multi-view U-Net group. It is denoted as Trans2U-Net. Extensive experiments on three datasets demonstrate the effectiveness of the proposed method.