Pemanfaatan Mediapipe Body Pose Estimation dan Dynamic Time Warping untuk Pembelajaran Tari Remo

Authors

  • Yusuf Effendi Informatic Department, Institut Sains dan Teknologi Terpadu Surabaya
  • Yosi Kristian Informatic Department, Institut Sains dan Teknologi Terpadu Surabaya
  • Lukman Zaman P.C.S.W Informatic Department, Institut Sains dan Teknologi Terpadu Surabaya
  • Hariadi Yutanto Informatic Department, Universitas Hayam Wuruk Perbanas Surabaya

DOI:

https://doi.org/10.26905/jtmi.v9i2.10408

Keywords:

Mediapipe, DTW, Tari Remo, Computer vision

Abstract

Video can be used as a learning medium for various purposes. In this research, the object of study is the movements of the traditional dance "remo." Thus, as a substitute for an absent coach or instructor, videos can take on the role of a dance instructor. However, video communication is one-way between the coach and the learners. Without movement correction, individuals trying to learn remo dance may find it challenging to determine if they are performing the movements correctly. Therefore, the author aims to create a system to assist coaches in correcting the dance movements of their learners. Using the MediaPipe module and the Dynamic Time Warping algorithm, the author developed a system to correct the learners' dance movements. This system can detect deviations from the coach's instructional video and provide notifications about which body angles do not match the coach's video instructions. The system operates by having users upload a video of their dance movements, and then it identifies which movements deviate from the correct remo dance. The accuracy is measured by comparing the angle distances between the master's movements and the test data. If the angle exceeds a predetermined threshold, the movement is considered incorrect. The system's output is validated by the coach, and it achieves 90% accuracy in identifying movement errors in videos. With this accuracy, the system can assist coaches in evaluating their learners' remo dance movements.

References

Viral food travel, “Mengenal Gerakan Tari Remo dan Sikap Dasarnya,†https://kumparan.com/, 2021. https://kumparan.com/viral-food-travel/mengenal-gerakan-tari-remo-dan-sikap-dasarnya-1vaBKJCqPyB/full (accessed Jun. 10, 2023).

M. Hardi, “Tari Remo dari Jawa Timur: Asal-Usul, Makna, dan Komposisinya,†https://www.gramedia.com/. https://www.gramedia.com/literasi/tari-remo/ (accessed Nov. 10, 2022).

D. Brown and M. Ndleve, “Virtual Gym Instructor,†22nd South. Africa Telecommun. Networks Appl. Conf., no. September, pp. 246–251, 2019.

F. Zhang et al., “MediaPipe Hands: On-device Real-time Hand Tracking,†2020, [Online]. Available: http://arxiv.org/abs/2006.10214

J. W. Kim, J. Y. Choi, E. J. Ha, and J. H. Choi, “Human Pose Estimation Using MediaPipe Pose and Optimization Method Based on a Humanoid Model,†Appl. Sci., vol. 13, no. 4, 2023, doi: 10.3390/app13042700.

G. Dsouza, D. Maurya, and A. Patel, “Smart gym trainer using Human pose estimation,†2020 IEEE Int. Conf. Innov. Technol. INOCON 2020, pp. 4–7, 2020, doi: 10.1109/INOCON50539.2020.9298212.

“MediaPipe Pose,†https://google.github.io/. https://google.github.io/mediapipe/solutions/pose.html (accessed Nov. 28, 2022).

A. Halder and A. Tayade, “Real-time Vernacular Sign Language Recognition using MediaPipe and Machine Learning,†Int. J. Res. Publ. Rev., no. 2, pp. 9–17, 2021, [Online]. Available: www.ijrpr.com

S. N. Budiman, S. Lestanti, S. M. Evvandri, and R. K. Putri, “Pengenalan Gestur Gerakan Jari Untuk Mengontrol Volume Di Komputer Menggunakan Library Opencv Dan Mediapipe,†Antivirus J. Ilm. Tek. Inform., vol. 16, no. 2, pp. 223–232, 2022.

T. K. Anggita Wijayanti Nurul; Suciati, Nanik, “Pengenalan Bahasa Isyarat Indonesia dengan Metode Dynamic Time Warping (DTW) menggunakan Kinect 2.0,†J. Tek. ITS, vol. 7, no. Vol 7, No 1 (2018), pp. 199–202, 2018, [Online]. Available: http://ejurnal.its.ac.id/index.php/teknik/article/view/28943

C. Dinata, D. Puspitaningrum, and E. Erna, “Implementasi Teknik Dynamic Time Warping (Dtw) Pada Aplikasi Speech To Text,†J. Tek. Inform., vol. 10, no. 1, pp. 49–58, 2018, doi: 10.15408/jti.v10i1.6816.

W. Ahmed, K. Chanda, and S. Mitra, “Vision based Hand Gesture Recognition using Dynamic Time Warping for Indian Sign Language,†Proc. - 2016 Int. Conf. Inf. Sci. ICIS 2016, pp. 120–125, 2017, doi: 10.1109/INFOSCI.2016.7845312.

Downloads

Additional Files

Published

2023-12-27

Issue

Vol. 9 No. 2 (2023): Desember 2023

Section

Articles

License

Authors who publish with this journal agree to the following terms:

(1) Copyright of the published articles will be transferred to the journal as the publisher of the manuscripts. Therefore, the author confirms that the copyright has been managed by the journal.

(2) Publisher of JTMI: Jurnal Teknologi dan Manajemen Informatika is University of Merdeka Malang.

(3) The copyright follows Creative Commons Attribution–ShareAlike License (CC BY SA): This license allows to Share — copy and redistribute the material in any medium or format, Adapt — remix, transform, and build upon the material, for any purpose, even commercially.