Designing a Facial Expression Detection System to Determine the Level of Customer Satisfaction with K-Nearest Neighbor Method

Diana Rahmawati, Koko Joni, Muhammad Iqbal Maulana


Facial expressions are one of the ways humans communicate to convey one's emotions to their communication partner nonverbally. Therefore, human facial expressions can be used for various purposes, one of which is knowing customer satisfaction. So far, customers of Bank Rakyat Indonesia (BRI) provide feedback on service quality using only a polling system, namely by filling out a criticism and suggestion form and then entering it in a suggestion box which is distinguished between satisfied and dissatisfied. However, such a method is less effective because it can be easily manipulated by customers and customers are often indifferent to the feedback. So that the improvement of service quality tends to be less effective. This research will design a system that can recognize human facial expressions to determine the level of customer satisfaction with input data in the form of video data taken by a webcam camera with the viola-jones method to detect faces and determine facial patterns. Then the facial data will be classified using the K-Nearest Neighbor method to determine the type of facial expression. Determination of the value of k will determine the success rate of facial expression detection. The processed data will be displayed on a liquid crystal display (LCD) and then stored in a MySQL database. The results showed that the accuracy of facial expression detection was 80.77% from 52 facial expression data.


Conventional System; Facial Expression; Customer Satisfaction; Viola-Jones; K-Nearest Neighbor

Full Text:



S. Normasari, S. Kumadji, and A. Kusumawati, “Pengaruh Kualitas Pelayanan Terhadap Kepuasan Pelanggan, Citra Perusahaan dan Loyalitas Pelanggan,” vol. 6, no. 2, pp. 1–9, 2013.

N. Aminarto, Y. A. Sari, and R. C. Wihandika, “Pengenalan Emosi Berdasarkan Ekspresi Mikro Menggunakan Metode Local Binary Pattern,” J. Pengemb. Teknol. Inf. dan Ilmu Komput., vol. 2, no. 10, pp. 3230–3238, 2018.

J. E. Prawitasllri, “Mengenal Emosi Melalui Komunikasi Nonverbal,” Bul. Psikol., vol. 3, no. 1, pp. 27–43, 2016, doi: 10.22146/bpsi.13384.

Y. F. Nadhiroh, “PENGENDALIAN EMOSI (Kajian Religio-Psikologis tentang Psikologi Manusia),” J. SAINTIFIKA Islam., vol. 2, no. 1, pp. 53–63, 2015.

E. Setyati, Y. K. Suprapto, and M. H. Purnomo, “Facial emotional expressions recognition based on active shape model and radial basis function network,” CIMSA 2012 - 2012 IEEE Int. Conf. Comput. Intell. Meas. Syst. Appl. Proc., pp. 41–46, 2012, doi: 10.1109/CIMSA.2012.6269607.

P. Viola, M. Jones, and M. Energy, “Robust Real-Time Face Detection Intro to Face Detection,” Int. J. Comput. Vis., vol. 57, no. 2, pp. 137–154, 2004.

M. H. Purnomo and A. Muntasa, Konsep Pengolahan Citra Digital dan Ekstraksi Fitur. Yogyakarta: Graha Ilmu, 2010.

D. Rahmawati, M. P. P. I, M. Ulum, and M. Ulum, “Identification and Classification of Pathogenic Bacteria Using the K-Nearest Neighbor Method Identifikasi Dan Klasifikasi Bakteri Patogen Dengan,” vol. 5, no. 1, 2021.

D. Purnama, Pengantar Machine Learning Konsep dan Praktikum Dengan Contoh Latihan Berbasis R dan PYTHON. Bandung: INFORMATIKA, 2019.

I. G. Rai et al., “Ekstraksi Fitur Warna , Tekstur dan Bentuk untuk Clustered-Based Retrieval of Images ( CLUE ),” vol. 16, no. April, 2017.



  • There are currently no refbacks.

JEEMECS (Journal of Electrical Engineering, Mechatronic and Computer Science)
Electrical Engineering Department, Faculty of Engineering

Mailling Address:

  • Address: Taman Agung Street No. 1, Sukun, Malang City, East Java, 65146, Indonesia.
  • Website:
  • Phone: +62 831 - 1205 - 2815 
  • Email:

JEEMECS (Journal of Electrical Engineering, Mechatronic and Computer Science) is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License Creative Commons License

Copyright ©2020 University of Merdeka Malang Powered by Open Journal Systems.