Continuous Sign Language Recognition Using Deep Learning Models
Open Access
Author:
Rahman, Rayhan
Area of Honors:
Computer Engineering
Degree:
Bachelor of Science
Document Type:
Thesis
Thesis Supervisors:
Mahanth Gowda, Thesis Supervisor John Morgan Sampson, Thesis Honors Advisor
Keywords:
deep learning DGS sign language continuous sign language recognition LSTM Encoder Encoder-Decoder RNN CNN WER RWTH-PHOENIX HMM CTC Loss Cross-Entropy Loss Softmax Function Loss Function
Abstract:
Continuous sign language recognition is inherently a difficult task. Unlike translation of traditional auditory language, sign language is a visual-spatial language which requires processing of significantly more information. Developers have been attempting to solve this problem for decades to help the nearly 430 million people around the world who suffer from hearing impairment [1]. The most common technical method of tackling this problem is to use a machine learning model that has been trained on a robust dataset. A number of prominent models and algorithms exist that can be applied on videos of sign language to learn and predict the transcription of words and sentences in sign language with varying rates of success, measured through word error rate (WER). This paper explores two different deep learning models of continuous sign language recognition – an encoder model paired with a connectionist temporal classification loss function and an encoder-decoder model with a cross-entropy loss function – to investigate which model produces a lower word error rate and why.
