TranscoderGAN: Transformer-Inception Based Encoder-Decoder Generative Adversarial Network for Thermal-Visible Face Transformation

This paper presents a novel transformer-guided encoder-decoder network for facial Image translation by using multi-layer perceptron and various depth levels. The proposed network exploits the advantage of multi-head self-attention mechanisms for efficient learning. Inspired by the generative capabilities of the generative adversarial network (GAN) for image-to-image translation, we employed the proposed Transformer-Inception based Encoder in the GAN framework. The proposed encoder-decoder network is used as a generator network. The proposed TranscoderGAN model facilitates the intense cyclic training for image-to-image translation efficiently by utilizing inception-based learning in residual networks. The proposed TranscoderGAN is used for facial image synthesis over the sketch in the CUHK dataset as well as the thermal faces of the WHU-IIP dataset. The proposed TranscoderGAN outperforms the state-of-the-art GAN methods for facial image translation. In quantitative results analysis, on an average { 10.36 } improvement is found over Structural Similarity Index Measure (SSIM) and Visual Information Fidelity (VIF), respectively, while { 55.38 } reduction over the Fréchet Inception Distance (Fid) for the CUHK dataset. For WHU-IIP thermal face dataset, the average improvement by the proposed model is { 9.63 %, 1.14 %} over SSIM, VIF, and { 89.25 %} reduction in Fid metrics, respectively.