Sign Language-based Chat System Using Android Application

Hearing loss, commonly referred to as hearing impairment, involves a partial or complete inability to hear. Deaf people find it very difficult to converse with those who are not aware of sign language. Communication between deaf and non-deaf individuals remains a significant challenge. To address this challenge, we utilized two approaches: the first one integrates MediaPipe with an Artificial Neural Network (ANN), and the second one employs a Convolutional Neural Network (CNN). The second approach yields better results. However, in live application, the first approach (MediaPipe and ANN) performs well in a noisy background. With the use of artificial intelligence, we convert American Sign Language into standard English for non-deaf people and vice-versa so that the deaf can also comprehend it. Experimental observations show that he achieved accuracy is 90.5 percentage and 92 percentage by using MediaPipe + ANN and CNN respectively. We also developed a real-time chat application that integrates these models, making everyday communication easier and more inclusive for people with hearing impairments.