Echoes in Silence: A Technological Leap for Pakistan Sign Language Translation and Recognition

In an endeavor to bridge communication barriers for the Pakistani deaf community, this project presents a robust system capable of translating Pakistani Sign Language (PSL) to text and vice versa. The architecture leverages cutting-edge technologies: the textual and voice inputs are processed using Natural Language Processing (NLP) techniques, with voice-to-text translation facilitated by the WebSpeechToolkit. For the Sign-to-Text (S2T) module, images and videos undergo rigorous preprocessing using OpenCV, followed by hand landmark detection via MediaPipe, ensuring the precise capture of PSL gestures. TensorFlow's neural networks are instrumental in recognizing these gestures, translating them into textual counterparts. The project's unique feature is its flexibility in breaking down non-existent sentences into words and, if needed, further into alphabets. Despite its innovative approach, the system acknowledges limitations in its training data, encompassing a dataset of English words, numbers, Urdu alphabets, and English alphabets, yet still achieving an impressive accuracy of approximately 95 %. Hosted on a user-friendly Django website interface, this system introduces two primary modules: one for translating text to PSL signs and another for the inverse. With its amalgamation of technologies and user-centric design, this project holds promise as a pivotal tool for enhancing the accessibility and interaction of the deaf community in Pakistan.