Pemanfaatan Mediapipe Body Pose Estimation dan Dynamic Time Warping untuk Pembelajaran Tari Remo

Yusuf Effendi; Yosi Kristian; Lukman Zaman P.C.S.W; Hariadi Yutanto

doi:10.26905/jtmi.v9i2.10408

Pemanfaatan Mediapipe Body Pose Estimation dan Dynamic Time Warping untuk Pembelajaran Tari Remo

Yusuf Effendi, Yosi Kristian, Lukman Zaman P.C.S.W, Hariadi Yutanto

Abstract

Video can be used as a learning medium for various purposes. In this research, the object of study is the movements of the traditional dance "remo." Thus, as a substitute for an absent coach or instructor, videos can take on the role of a dance instructor. However, video communication is one-way between the coach and the learners. Without movement correction, individuals trying to learn remo dance may find it challenging to determine if they are performing the movements correctly. Therefore, the author aims to create a system to assist coaches in correcting the dance movements of their learners. Using the MediaPipe module and the Dynamic Time Warping algorithm, the author developed a system to correct the learners' dance movements. This system can detect deviations from the coach's instructional video and provide notifications about which body angles do not match the coach's video instructions. The system operates by having users upload a video of their dance movements, and then it identifies which movements deviate from the correct remo dance. The accuracy is measured by comparing the angle distances between the master's movements and the test data. If the angle exceeds a predetermined threshold, the movement is considered incorrect. The system's output is validated by the coach, and it achieves 90% accuracy in identifying movement errors in videos. With this accuracy, the system can assist coaches in evaluating their learners' remo dance movements.

Keywords

Mediapipe; DTW; Tari Remo; Computer vision

Full Text:

PDF

References

Viral food travel, “Mengenal Gerakan Tari Remo dan Sikap Dasarnya,” https://kumparan.com/, 2021. https://kumparan.com/viral-food-travel/mengenal-gerakan-tari-remo-dan-sikap-dasarnya-1vaBKJCqPyB/full (accessed Jun. 10, 2023).

M. Hardi, “Tari Remo dari Jawa Timur: Asal-Usul, Makna, dan Komposisinya,” https://www.gramedia.com/. https://www.gramedia.com/literasi/tari-remo/ (accessed Nov. 10, 2022).

D. Brown and M. Ndleve, “Virtual Gym Instructor,” 22nd South. Africa Telecommun. Networks Appl. Conf., no. September, pp. 246–251, 2019.

F. Zhang et al., “MediaPipe Hands: On-device Real-time Hand Tracking,” 2020, [Online]. Available: http://arxiv.org/abs/2006.10214

J. W. Kim, J. Y. Choi, E. J. Ha, and J. H. Choi, “Human Pose Estimation Using MediaPipe Pose and Optimization Method Based on a Humanoid Model,” Appl. Sci., vol. 13, no. 4, 2023, doi: 10.3390/app13042700.

G. Dsouza, D. Maurya, and A. Patel, “Smart gym trainer using Human pose estimation,” 2020 IEEE Int. Conf. Innov. Technol. INOCON 2020, pp. 4–7, 2020, doi: 10.1109/INOCON50539.2020.9298212.

“MediaPipe Pose,” https://google.github.io/. https://google.github.io/mediapipe/solutions/pose.html (accessed Nov. 28, 2022).

A. Halder and A. Tayade, “Real-time Vernacular Sign Language Recognition using MediaPipe and Machine Learning,” Int. J. Res. Publ. Rev., no. 2, pp. 9–17, 2021, [Online]. Available: www.ijrpr.com

S. N. Budiman, S. Lestanti, S. M. Evvandri, and R. K. Putri, “Pengenalan Gestur Gerakan Jari Untuk Mengontrol Volume Di Komputer Menggunakan Library Opencv Dan Mediapipe,” Antivirus J. Ilm. Tek. Inform., vol. 16, no. 2, pp. 223–232, 2022.

T. K. Anggita Wijayanti Nurul; Suciati, Nanik, “Pengenalan Bahasa Isyarat Indonesia dengan Metode Dynamic Time Warping (DTW) menggunakan Kinect 2.0,” J. Tek. ITS, vol. 7, no. Vol 7, No 1 (2018), pp. 199–202, 2018, [Online]. Available: http://ejurnal.its.ac.id/index.php/teknik/article/view/28943

C. Dinata, D. Puspitaningrum, and E. Erna, “Implementasi Teknik Dynamic Time Warping (Dtw) Pada Aplikasi Speech To Text,” J. Tek. Inform., vol. 10, no. 1, pp. 49–58, 2018, doi: 10.15408/jti.v10i1.6816.

W. Ahmed, K. Chanda, and S. Mitra, “Vision based Hand Gesture Recognition using Dynamic Time Warping for Indian Sign Language,” Proc. - 2016 Int. Conf. Inf. Sci. ICIS 2016, pp. 120–125, 2017, doi: 10.1109/INFOSCI.2016.7845312.

DOI: https://doi.org/10.26905/jtmi.v9i2.10408