Image fusion algorithm based on multi-scale detail siamese convolutional neural network

Image fusion is to fuse the complementary images under different acquisition conditions or sensors for improving the visual quality of images. For this task, this paper proposes an improved multi-scale fusion algorithm based on the combination of rolling guided filtering and neural network. Firstly, the siamese convolutional neural network is used to learn the image features and further obtain the weight map containing the significant features of the source image. Then, the image is decomposed by the improved rolling guide filter, and the weight parameters of the rolling guide filter are adaptive to achieve multi-scale adaptive decomposition by combining the information entropy, and the details of the image are enhanced by combining the nonlinear mapping. Finally, based on the local energy and weight map, an adaptive fusing strategy is designed to combine the multi-scale images adaptively. The experimental comparison shows that the proposed method can avoid the halo effect of image edge and better highlight the edge and detail texture features of image. In addition, compared with other algorithms, our algorithm can achieve a better performance according to the objective evaluation index items, such as average gradient, information entropy, visual information fidelity and spatial frequency.