Image Captioning Using Vision Transformer Encoder Decoder Model

The automated generation of a NLP of an image has been in the spotlight because it is important in real-world applications and because it involves two of the most critical subfields of artificial intelligence, computer vision and natural language processing. The image captioning process involves generating descriptive explanations of the content and activities contained within an image. Image Captioning is an approach that employs descriptive language to make explanations of images. Image captioning is an extremely useful tool in many applications, including the analysis of large sets of unlabeled photographs and discovering hidden patterns, which are of great importance in machine learning applications. Besides, image captioning is also crucial in the creation of software used to navigate autonomous cars and also for assisting visually impaired people. The Image Captioning task can be achieved through the use of Deep Learning Models. The achievements in deep learning and Natural Language Processing have made it possible to create captions for images. This research utilizes Neural Networks as an approach to image captioning. The Convolutional Neural Network (ResNet) is used as an encoder, which removes the features of the pictures. The Recurrent Neural Network (Long Short Term Memory) acts as a decoder, creating captions for the images based on the removed image features and a built vocabulary.