SIBI (Sistem Bahasa Isyarat Indonesia) berbasis Machine Learning dan Computer Vision untuk Membantu Komunikasi Tuna Rungu dan Tuna Wicara

Saiful Nur Budiman; Sri Lestanti; Haris Yuana; Beta Nurul Awwalin

doi:10.26905/jtmi.v9i2.10993

SIBI (Sistem Bahasa Isyarat Indonesia) berbasis Machine Learning dan Computer Vision untuk Membantu Komunikasi Tuna Rungu dan Tuna Wicara

Saiful Nur Budiman, Sri Lestanti, Haris Yuana, Beta Nurul Awwalin

Abstract

The Indonesian Sign Language System (SIBI) is used to translate sign language into text or speech. SIBI helps improve communication between people using sign language and those who do not understand it. Unlike commonly used languages, SIBI sign language is less known to most people due to a lack of interest. To address this, an artificial intelligence-based application was developed, focusing on deep learning to recognize SIBI sign language hand movements in real-time. The model was created with 20 epochs, a batch size of 16, and a learning rate of 0.001. It consists of 13 layers, with the ReLU activation function used for the input layer, while the output layer uses Sigmoid. The ADAM optimizer was used to expedite the model creation process. The image dataset used had a size of 300x300 pixels. In the classification testing of the SIBI alphabet results in this study, it was tested using distance tests. The distance between the webcam and the SIBI language speaker was divided into two categories: 40 cm and 60 cm. For a 40-cm distance, an accuracy of 87.50% was obtained, and for a 60-cm distance, an accuracy of 79.17% was achieved. One limitation of this study is that two alphabets, J and Z, were not included in the dataset. This is because recognition of these two alphabets requires not only finger pattern recognition but also recognition of their gesture patterns.

Keywords

Sign Language (SIBI); Deep Learning; Computer Vision;

Full Text:

PDF

References

B. Rahmat and B. Nugroho, Pemrograman Deep Learning dengan Python. Indomeida Pustaka, 2021.

D. Prasetyawan and R. Gatra, "Model Convolutional Neural Network untuk Mengukur Kepuasan Pelanggan Berdasarkan Ekspresi Wajah," Jurnal Teknik Informatika dan Sistem Informasi (JuTISI), vol. 8, 2022.

R. Daroya, D. Peralta, and P. Naval, "Alphabet Sign Language Image Classification Using Deep Learning," presented at the Conf. Proceedings TENCON, 2018.

Darmatasia, "Pengenalan Sistem Isyarat Bahasa Indonesia (SIBI) Menggunakan Gradient-Convolutional Neural Network," Jurnal Informatika Sains dan Teknologi (INSTEK), vol. 6, 2021.

S. Dwijayanti, Hermawati, S. I. Taqiyyah, H. Hikmarika, and B. Y. Suprapto, "Indonesia Sign Language Recognition using Convolutional Neural Network," International Journal of Advanced Computer Science and Applications (IJACSA), vol. 12, 2021.

V. R. S. Nastiti, R. A. Muhammad, and B. P. Putra, "Pendeteksi Bahasa Isyarat Gestur Tangan dengan Menggunakan Model CNN ResNet50V2," Rekayasa Sistem dan Teknologi Informasi (RESTI), vol. 6, 2022.

H. P. A. Tjahyaningtijas, W. Yustanti, and A. Prihanto, "Analisa Learning rate dan Batch size Pada Klasifikasi Covid Menggunakan Deep learning dengan Optimizer ADAM," Journal Information Engineering and Educational Technology (JIEET), vol. 5, 2021.

F. Zhang et al., "MediaPipe Hands:On-device Real-time Hand Tracking," presented at the CVPR Workshop on Computer Vision for Augmented and Virtual Reality, Seattle, USA, 2020.

Indriani, M. Harris, and A. S. Agoes, "Applying Hand Gesture Recognition for User Guide Application Using MediaPipe," in Proceedings of the 2nd International Seminar of Science and Applied Technology (ISSAT 2021), 2021, vol. 207.

M. K. Hossen and M. S. Uddin, "A dataset for Assessing Real-time Aattention Levels of the Students During Online Classes," Data in Brief, vol. 51, 2023.

P. A. Nugroho, I. Fenriana, and R. Arijanto, "Implementasi Deep Learning Menggunakan Convolutional Neural Network (CNN) pada Ekspresi Manusia," ALGOR Journal, vol. 2, 2020.

M. Reyad, A. M.Sarhan, and M.Arafa, "A Modified ADAM Algorithm for Deep Neural Network Optimization," Neural Computing and Applications, vol. 35, pp. 17095-17112, 2023.

D. Irfan, R. Rosnelly, M. Wahyni, J. T. Samudra, and A. Rangga, "Perbandingan Optimasi SGD, ADADELTA dan ADAM Dalam Klasifikasi Hydrangea Menggunakan CNN," Journal of Science and Social Research, vol. 5, pp. 244-253, 2022.

F. D. Telaumbanua, P. Hulu, T. Z. Nadeak, R. R. Lumbantong, and A. Dharma, "Penggunaan Machine Learning Di Bidang Kesehatan," Jurnal Teknologi dan Ilmu Komputer Prima (Jutikomp), vol. 2, 2020.

L. B. Ardianto, M. I. Wahyuddin, and W. Winarsih, "Implementasi Deep Learning untuk Sistem Keamanan Data Pribadi Menggunakan Pengenalan Wajah dengan Metode Eigenface Berbasis Android," Jurnal Teknologi Informasi dan Komunikasi (JTIK), vol. 5, 2021.

T. Israldi and E. H. S. S. F. Syafria, "Klasifikasi American Sign Language Menggunakan Convolutional Neural Network," Building of Informatics, Technology and Science (BITS), vol. 4, 2022.

DOI: https://doi.org/10.26905/jtmi.v9i2.10993