Convolutional Neural Network with Class-Dependent Label Smoothing Method for Indian Sign Language Recognition

Abstract

Sign language is a significant way of communication for individuals with hearing impairments. To bridge a communication gap between the hearing impaired and the general public, it is important to recognize a sign accurately. The recognition of sign language using hand gesture images is a challenging task due to similar gesture images in different classes, which minimizes recognition performance. The Convolutional Neural Network (CNN) with the Class Dependent Label Smoothing (CDLS) technique is being developed to recognize the Indian Sign Language by hand gesture images. The CDLS is a regularization technique that allows the method to assign smoothing values based on the characteristics of gestures in various classes and enhances the CNN performance for sign language recognition. The ResNet-50-based feature extraction technique is utilized to extract deep features of gestures to identify variant gestures in different classes. The CNN with the CDLS technique reaches 99.6% accuracy, 0.99% precision, 0.99% recall, and a 0.99% F1-score on the Indian Sign Language (ISL) dataset when compared to CNN1.