Klasifikasi Gerakan Bahasa Isyarat Indonesia (Bisindo) menggunakan Arsitektur Transfer Learning Xception

Meisya Vira  Amelia; Wahyu Syaifullah Jauharis Saputra; Kartika Maulida Hindrayani

doi:10.26905/jasiek.v7i2.15674

Authors

Meisya Vira Amelia Program Studi Sains Data, Fakultas Ilmu Komputer, Universitas Pembangunan Nasional "Veteran" Jawa Timur
Wahyu Syaifullah Jauharis Saputra Program Studi Sains Data, Fakultas Ilmu Komputer, Universitas Pembangunan Nasional "Veteran" Jawa Timur
Kartika Maulida Hindrayani Program Studi Sains Data, Fakultas Ilmu Komputer, Universitas Pembangunan Nasional "Veteran" Jawa Timur

DOI:

https://doi.org/10.26905/jasiek.v7i2.15674

Keywords:

BISINDO, Convolutional Neural Network, Sign Langauge, Transfer Learning, Xception

Abstract

Human communication generally relied on speech. However, this was not applicable to the deaf people, who depended on sign language for daily interactions. Unfortunately, not everyone had the ability to understand sign language. In higher education environments, the lack of individuals proficient in sign language often created inequality in the learning process for deaf students. This limitation could be addressed by fostering a more inclusive environment, one of which was through the implementation of a sign language translation system. Therefore, this study aimed to develop a machine learning model capable of detecting and translating Indonesian Sign Language (BISINDO) alphabet gestures. The model was built using the Xception transfer learning method from Convolutional Neural Networks (CNN). The dataset consisted of 26 BISINDO alphabet gestures with a total of 650 images. The model was evaluated using K-Fold cross-validation and achieved an F1-score of 98% during testing

Downloads

Download data is not yet available.

References

T. M. Milyane and Dkk, Pengantar Ilmu Komunikasi, vol. 5, no. 3. 2022. [Online]. Available: https://repository.penerbitwidina.com/media/publications/557082-pengantar-ilmu-komunikasi-22ec77af.pdf

[BPS, “Penduduk Menurut Wilayah dan Tingkat Kesulitan Mendengar,” Badan Pusat Statistik. Accessed: Sep. 13, 2024. [Online]. Available: https://sensus.bps.go.id/topik/tabular/sp2022/145/0/0

BPS, “Jumlah Penduduk Kesulitan Berbicara,” Badan Pusat Statistik. Accessed: Sep. 13, 2024. [Online]. Available: https://sensus.bps.go.id/topik/tabular/sp2022/146/1/0

LMD, “Sebaran Mahasiswa Disabilitas,” Layanan Mahasiswa Disabilitas. Accessed: Sep. 13, 2024. [Online]. Available: https://ptinklusif.kemdiktisaintek.go.id/s/5/sebaran-mahasiswa-disabilitas

Effranzudeta, Yesinovitasari, Herdi, and Dinafitriani, “Journal of Special Education Lectura Urgensi Bahasa Isyarat di Lingkungan Universitas,” vol. 2, no. 1, pp. 65–72, 2024.

W. S. J. Saputra, M. H. Fardana, and M. A. R. Valentino, “Implementasi AI Pendeteksi Pola Gerak Tangan pada Game ‘Pong Ball’ dengan Menggunakan Algoritma Convolutional Neural Network,” J. Manajamen Inform. Jayakarta, vol. 2, no. 3, p. 235, 2022, doi: 10.52362/jmijayakarta.v2i3.834.

D. A. Prasetya, A. P. Sari, P. A. Riyantoko, and T. M. Fahrudin, “The Effect of Information Quality and Service Quality on User Satisfaction of the Government of Kabupaten Malang,” TIERS Inf. Technol. J., vol. 4, no. 1, pp. 32–42, 2023, doi: 10.38043/tiers.v4i1.4328.

T. M. Fahrudin, P. A. Riyantoko, K. M. Hindrayani, and E. M. Safitri, “An Introduction To Machine Learning Games And Its Application For Kids In Fun Project,” Int. J. Comput. Netw. Secur. Inf. Syst., vol. 2, no. 1, pp. 26–30, 2020, [Online]. Available: https://machinelearningforkids.co.uk

M. Idhom, D. A. Prasetya, P. A. Riyantoko, T. M. Fahrudin, and A. P. Sari, “Pneumonia Classification Utilizing VGG-16 Architecture and Convolutional Neural Network Algorithm for Imbalanced Datasets,” TIERS Inf. Technol. J., vol. 4, no. 1, pp. 73–82, 2023, doi: 10.38043/tiers.v4i1.4380.

S. Shania, M. Farid Naufal, V. Riandaru Prasetyo, and M. S. Bin Azmi, “Translator of Indonesian Sign Language Video using Convolutional Neural Network with Transfer Learning,” Indones. J. Inf. Syst., vol. 5, no. 1, pp. 17–27, 2022, doi: 10.24002/ijis.v5i1.5865.

I. D. A. Rachmawati, R. Yunanda, M. F. Hidayat, and P. Wicaksono, “Deep Transfer Learning for Sign Language Image Classification: A Bisindo Dataset Study,” Eng. Math. Comput. Sci. J., vol. 5, no. 3, pp. 175–180, 2023, doi: 10.21512/emacsjournal.v5i3.10621.

M. Sari and E. R. Jamzuri, “Hand Sign Recognition of Indonesian Sign Language System ( SIBI ) Using,” vol. 5, no. 158, pp. 258–265, 2026.

F. Chollet, “Xception: Deep Learning with Depthwise Separable Convolutions,” Proc. IEEE Conf. Comput. Vis. pattern Recognit., vol. 7, no. 3, pp. 1251–1258, 2017, doi: https://doi.org/10.48550/arXiv.1610.02357.

A. Noer, “Bahasa Isyarat Indonesia (BISINDO) Alphabets.” [Online]. Available: https://www.kaggle.com/datasets/achmadnoer/alfabet-bisindo

A. Muhaimin, W. Wibowo, and P. A. Riyantoko, “Multi-label Classification Using Vector Generalized Additive Model via Cross-Validation,” J. Inf. Commun. Technol., vol. 22, no. 4, pp. 657–673, 2023, doi: https://doi.org/10.32890/jict2023.22.4.5.

R. Tuntun, K. Kusrini, and K. Kusnawi, “Analisis Perbandingan Kinerja Algoritma Klasifikasi dengan Menggunakan Metode K-Fold Cross Validation,” J. Media Inform. Budidarma, vol. 6, no. 4, p. 2111, 2022, doi: 10.30865/mib.v6i4.4681.