Indian sign language recognition system using network deconvolution and spatial transformer network

A sign language recognition system can be applied to reduce a communication gap between deaf and normal persons. However, the Indian sign language recognition (ISL) systems are in the developing stage. Most of the recent ISL recognition systems use convolutional neural networks (CNNs) where applied convolution operation shifts a kernel to overlapping portions over the image. However, these kernels may learn redundant data since real-world images have very high correlations. The training process of neural networks is challenging for redundant image data. To overcome this limitation, an ISL recognition system has been proposed in this paper that uses the network deconvolution technique. This technique reduces not only pixel-wise correlation but also a channel-wise correlation in images. The proposed model is also augmented with a spatial transformer network to increase spatial invariance of convolution operations against spatial transformations. The proposed recognizer offers better accuracy most of the time than other experimented systems on two ISL datasets VUCS_ISL_I and created VUCS_ISL_II and standard datasets of other sign languages, i.e., American sign language, Arabic sign language, Spanish sign language.