3D-Transunet for Brain Metastases Segmentation in the BraTS2023 Challenge

Segmenting brain tumors is complex due to their diverse appearances andscales. Brain metastases, the most common type of brain tumor, are a frequentcomplication of cancer. Therefore, an effective segmentation model for brainmetastases must adeptly capture local intricacies to delineate small tumorregions while also integrating global context to understand broader scanfeatures. The TransUNet model, which combines Transformer self-attention withU-Net's localized information, emerges as a promising solution for this task.In this report, we address brain metastases segmentation by training the3D-TransUNet model on the Brain Tumor Segmentation (BraTS-METS) 2023 challengedataset. Specifically, we explored two architectural configurations: theEncoder-only 3D-TransUNet, employing Transformers solely in the encoder, andthe Decoder-only 3D-TransUNet, utilizing Transformers exclusively in thedecoder. For Encoder-only 3D-TransUNet, we note that Masked-Autoencoderpre-training is required for a better initialization of the Transformer Encoderand thus accelerates the training process. We identify that the Decoder-only3D-TransUNet model should offer enhanced efficacy in the segmentation of brainmetastases, as indicated by our 5-fold cross-validation on the training set.However, our use of the Encoder-only 3D-TransUNet model already yield notableresults, with an average lesion-wise Dice score of 59.8% on the test set,securing second place in the BraTS-METS 2023 challenge.