A Multimodal Image Registration System for Histology Images

Histology image registration involves aligning microscopy images for various purposes, including creating 3D reconstructions from 2D, combining data from slices with different stain samples, and from multimodal registration. However, this process poses several challenges, including high resolution, non-linear elastic deformation, occlusions, missing sections, non-rigid deformation, contrast differences, and differences in appearance and local structure. Multimodal image registration is particularly challenging because different modalities may have different characteristics and require specific optimization algorithms. To address this issue, it is important to develop software that allows users to test different image registration algorithms and combine annotations made on them, in order to leverage the benefits of multiple modalities. To address this challenge, we developed a cloud-based Multimodal Image Registration system that enables developers and researchers to visually test the outcomes of various image registration algorithms. The system includes a project manager, an algorithm manager, and an image visualization system. The system was developed using the framework Django, JavaScript, and multiple libraries that facilitate the management and annotation of very high-resolution images. To demonstrate the effectiveness and flexibility of our system, we tested it using two different algorithms, SIFT and ORB, on nonlinear multimodal and brightfield images using the Hematoxylin and Eosin staining methods. The results show the system's ability to handle challenging image registration tasks while providing visualization tools to improve user experience.