Mahyoub, Mohamed and Natalia, Friska and Sudirman, Sud and Mustafina, Jamila (2023) Sign Language Recognition using Deep Learning. In: 2023 15th International Conference on Developments in eSystems Engineering (DeSE).
|
Text
Sign Language Recognition using Deep Learning.pdf Download (2MB) | Preview |
Abstract
Sign Language Recognition is a form of action recognition problem. The purpose of such a system is to automatically translate sign words from one language to another. While much work has been done in the SLR domain, it is a broad area of study and numerous areas still need research attention. The work that we present in this paper aims to investigate the suitability of deep learning approaches in recognizing and classifying words from video frames in different sign languages. We consider three sign languages, namely Indian Sign Language, American Sign Language, and Turkish Sign Language. Our methodology employs five different deep learning models with increasing complexities. They are a shallow four-layer Convolutional Neural Network, a basic VGG16 model, a VGG16 model with Attention Mechanism, a VGG16 model with Transformer Encoder and Gated Recurrent Units-based Decoder, and an Inflated 3D model with the same. We trained and tested the models to recognize and classify words from videos in three different sign language datasets. From our experiment, we found that the performance of the models relates quite closely to the model's complexity with the Inflated 3D model performing the best. Furthermore, we also found that all models find it more difficult to recognize words in the American Sign Language dataset than the others.
| Item Type: | Conference or Workshop Item (Paper) |
|---|---|
| Keywords: | Sign Language, Sign Language Recognition, Deep Learning, Convolutional Neural Networks |
| Subjects: | 000 Computer Science, Information and General Works > 000 Computer Science, Knowledge and Systems > 006 Special Computer Methods > 006.3 Artificial Intelligence, Machine Learning, Pattern Recognition, Data Mining 400 Language > 410 Linguistics > 419 Sign Languages |
| Divisions: | Faculty of Engineering & Informatics > Information System |
| Depositing User: | Administrator UMN Library |
| Date Deposited: | 07 Nov 2023 00:41 |
| Last Modified: | 07 Nov 2023 00:41 |
| URI: | https://kc.umn.ac.id/id/eprint/27049 |
Actions (login required)
 |
View Item |
