P-ISSN: 1979-5661

E-ISSN: 2622-321X

Jurnal Ilmu Komputer Vol 16 Nomor 2

Implementation of Face Recognition for Attendance Recording in Online Learning

I Dewa Made Bayu Atmaja Darmawan

Department of Informatics, Universitas Udayana

Bali, Indonesia

dewabayu@unud.ac.id

Abstract

The utilization of facial recognition technology has become increasingly imperative within the realm of online learning. The current study introduces a novel system that utilizes face recognition technology to record attendance in online learning environments. The attendance system necessitates students to activate an attendance button, whereby their attendance is subsequently documented through facial recognition technology. The system recognizes students as present solely based on facial recognition. The system stores the duration of online learning activities in a database. Implementing machine learning methodologies, specifically face detection algorithms, improves precision and efficacy in administering student attendance in online education. The system utilizes Haar cascades in OpenCV to detect faces, extract features such as eyes, nose, and mouth, and classify them using LBPH. Through extensive experiments, an accuracy rate of 93.55% was achieved. The study demonstrates the effectiveness of the combined approach, showcasing the potential of Haar cascades and LBPH in face recognition tasks. The present study makes a valuable contribution to the domains of computer vision and educational technology by offering a pragmatic remedy for attendance tracking in virtual learning settings.

Keywords: face detection, online learning, attendance recording, machine learning, opencv

  • 1.    Introduction

In recent years, the rise of online learning platforms has revolutionized the way education is delivered. However, one of the challenges in online learning is ensuring accurate attendance recording. Traditional methods such as manual attendance tracking can be time-consuming and prone to errors. To address this issue, the implementation of face recognition technology has emerged as a promising solution.

Various researchers have explored the use of deep learning convolutional neural networks (CNNs) for facial recognition [1]–[3]. Transfer learning, a technique that involves using pre-trained CNNs and training them on specific data, has been employed to improve the accuracy of facial recognition systems [1]. Additionally, researchers have explored the use of machine learning algorithms, such as Local Binary Pattern Histogram (LBPH), to enhance the classification performance of face recognition systems [1], [4]. LBPH (Local Binary Patterns Histograms) classifier is a type of algorithm used in face recognition systems. It combines the concepts of Local Binary Patterns (LBP) and histograms to perform face recognition tasks. On the other hand, deep learning models, such as convolutional neural networks (CNNs), have shown remarkable performance in various computer vision tasks, including image classification. Deep learning models excel in learning hierarchical representations from raw data, automatically extracting relevant features for classification.

LBPH classifiers have been used in various research studies and applications. They have been combined with other techniques such as Convolutional Neural Networks (CNN) for improved feature extraction and classification [5]. LBPH-based face recognition systems have been proposed for different purposes, including enhancing real-time face recognition and developing systems for visually impaired individuals [6], [7]. The LBPH classifier is implemented in OpenCV and can be trained using labeled face images to recognize and classify faces [8].

The choice of LBPH as the classifier for remote sensing image classification is motivated by its ability to handle limited training data, effectively model non-linear relationships, provide interpretability, and offer computational efficiency. These advantages make LBPH a suitable choice in scenarios where the dataset size is small, non-linear relationships exist, interpretability is desired, and computational resources are limited.

This paper focuses on the implementation of face recognition for attendance recording in the context of online learning. The objective is to develop a robust and efficient system that can automatically recognize and record students' attendance based on their facial features. The utilization of OpenCV, a popular computer vision library, in combination with the Haar cascade and LBPH methods, enables the detection and recognition of faces with satisfactory accuracy. Face recognition involves identifying and verifying the identity of individuals based on their facial characteristics. The Haar cascade method, pioneered by Viola and Jones, is utilized for face detection, enabling the system to detect human faces in images or video streams. The detected faces are then processed using the LBPH classifier, which performs classification based on a trained model.

The proposed system aims to overcome the challenges associated with attendance recording in online learning environments. By leveraging face recognition technology, the system eliminates the need for manual attendance tracking and provides a more efficient and reliable method for recording attendance.

The remainder of this paper is organized as follows. Section 2 provides an overview of the related work and existing approaches in the field of face recognition for attendance recording. Section 3 presents the methodology employed, including the implementation of the Haar cascade and LBPH methods using OpenCV. Section 4 discusses the experimental setup and presents the results and evaluation of the implemented system. Finally, Section 5 concludes the paper and discusses future directions for research and improvements.

  • 2.    Reseach Methods

The study focuses on the implementation of face recognition for attendance recording in the context of online learning. To achieve this, the researchers employ a combination of methodologies, including computer vision and machine learning techniques. Specifically, the Haar cascade method, pioneered by Viola and Jones [8], [9], is utilized for face detection, and the Local Binary Pattern Histogram (LBPH) classifier is employed for face recognition.

The Haar cascade method is a popular approach for detecting objects in images or video streams, particularly human faces. It involves training a classifier with positive and negative samples to identify specific patterns and features associated with the target object. In this case, the Haar cascade method enables the system to detect human faces accurately [10].

Furthermore, the LBPH classifier is utilized for face recognition, which involves identifying and verifying the identity of individuals based on their facial characteristics. LBPH is a machine learning algorithm that aims to find an optimal hyperplane in a high-dimensional feature space, enabling effective separation of different classes of data points. In this study, LBPH performs the classification based on a trained model to recognize and record students' attendance [10].

By combining the Haar cascade method for face detection and the LBPH classifier for face recognition, the researchers aim to develop a robust and efficient system for automatic attendance recording in online learning environments. These research methods leverage computer vision techniques and machine learning algorithms to address the challenge of accurate attendance tracking in online learning, providing a more efficient and reliable method for recording attendance.

Figure 1 shows each stage of the framework being built. The stage begins with capturing frames using a webcam, detecting faces in the frames, then facial image fragments are detected in a machine learning model. The results of face detection indicating that the learner is still online are recorded in a centralized database.

Figure 1. System Architecture of Face Recognition-based Attendance Recording

  • 3.    Result and Discussion

In this section, a discussion of system design, machine learning modeling, and accuracy testing of the built model will be carried out. In this section, a discussion of system design, machine learning modeling, and accuracy testing of the built model will be carried out. The system is built using the Python programming language and uses the Flask framework to build a web-based system. The system was built using the OpenCV library to implement the face detection method with Haar Cascade and classification using LBPH.

  • 3.1.    Result

Data gathering is carried out when the user/student registers on the system. In addition to providing the student's personal identity, 100 frames of facial images are captured for 5 seconds. The image capture process described earlier also performs a face detection process which produces 100 frames with cropped faces. The 100 facial frames generated in the image capture are used as training data in machine learning. Figure 2 shows an example of some of the captured images.

1001.234.ljp9 IKlWS-Zjpg 100l.236-3.jpg     10D1.2374.jpg     1001.23B-S.jpg     10□1,239.6.jpg

1001,249.16.jpg    1001.250.17.jpg    1W1.251.18.jpg    1001 -252.19.jpg    1001253.20jpg    1001.254.21.jpg

1001.264.31-jpg    1001.265.32.jpg    1001.266.33.jpg    1001.267 34.jpg l001.268.35.jpg    100l269.36.jpg

100 ∙27946jpg    1001.280.47.jpg    1M1.28148.jpg    1001.282.49.jpg    W01283.50.jpg    1001.284.51.jpg

1001.294.61.jpg    1001 _295.62.jpg     IE01296.63.jpg    100L297.64.jpg    10Ol298,65jpg    100l2S9.66.jpg

1001,309.76.jpg     10Q1.310.77.jpg     1001.311.7B.jpg     1001.312_79.jpg     1001.313.80.jpg     1001.314.B1.jpg

Figure 2. Result of Face Detection

There are six database tables to build a predefined functional system. Tables that store information about users consist of Student and Lecturer tables that store the identities of students and lecturers. There are six database tables to build a predefined functional system. Tables that store information about users consist of Student and Lecturer tables that store the identities of students and lecturers.

The class table records the courses attended by students or taught by a lecturer. The cardinality relationship between Student and Class is 1 to many, so it requires a new table called Learn. Meanwhile, because a lecturer can teach several classes, and there is a limitation that a class can only be taught by a lecturer, the lecturer id becomes a foreign key in the Class table. The face scanning process that is carried out continuously to calculate the learning duration is recorded in the acc_hist table. Finally, the img_dataset table stores information from frames which are the result of data gathering during the user registration process.

Figure 3. Database Schema

Machine learning modeling is done using the Python programming language. The web-based interface is built to make it easier for users to access the system and does not depend on a device platform. The Flask framework used to implement building web-based systems with Python programming implements the MVC concept. The MVC concept that organizes code implementation for Models, Views and Controller is an easy solution for integrating machine

learning models with a user-friendly interface. Figure 4 shows several examples of the user interface display of an online learning process attendance recording system.

(a)

(b)

Figure 4. User Interfaces, (a) Selecting Course, (b) Face Scanning

  • 3.2.    Discussion

A web-based online learning attendance recording system can be built using the Flask framework. Various educational institutions can use this system to record the duration of student self-learning based on face recognition. A teacher can create a class with an enrollment key and then ask students to register on the system. Then, students can sign up for a subject using the enrollment key given. The background of creating this system is to perform presence recordings during online learning activities, such as video conference learning activities. A teacher may need help observing the presence of all of his students because the number of classes could be too many to be able to perform on the screen. Thus, this system can be used for student presence recording based on face recognition. The system will record the student’s presence in front of the computer during the learning session.

Several barriers are concerned with implementing the presence recording system of online learning to be used as a suggestion for future research. A webcam can generally only be active for up to one application. When a student enables a webcam for a video conference app, the webcam resource will be locked for that application and cannot be used by other apps. This can be solved by using a virtual camera. Virtual cameras can be used as an adapter to the system, and initial image processing can be applied to improve the performance of the system. Another challenge is that the system cannot yet monitor processes running on the computer. A system may be able to remain in front of the computer but not follow the learning process because it is

redirected to applications or sites outside of the learning content. Adding features to record processes or sites that students are accessing can be considered, although consideration should be paid to the privacy aspects of students.

Accuracy testing starts with 5 users and is added every 5 multiples to a maximum of 30 users. The test method is applied to face detection with the Haar Cascade method and face recognition with the LBPH method. The face detection test is performed by running the registration feature. The results of the face detection test showed that the entire experiment Haar Cascade method could detect the face area correctly. Next, a face recognition test is performed by running the “start learning” feature for 10 seconds. Table 1 shows the results of the face recognition test on 30 users. The highest average accuracy occurred on the number of registered users of 5 users with 93.54%. Although there was a relative decrease in accuracy in the increase in the number of registered users, the ranking was not significant. Some things that can affect the accuracy of LBPH include lighting conditions, pose variation, occlusions, and dataset size/diversity.

Table 1. Accuracy Measurement on Face Recognition with LBPH

Num. of User Accuracy

5

93.55

10

90

15

91.7

20

89.79

25

90.34

30

85.15

  • 4.    Conclusion

The combination of Haar Cascade and LBPH methods can be applied to build an online presence recording system based on face recognition with a maximum accuracy of 93.55%. The system is built using the Flask framework that implements the MVC concept. It makes it easy for developers to build a user-friendly system without leaving OpenCV support available in Python. There are functional and virtual camera features that can be developed to improve the functionality of the system. Applying deep learning methods such as CNN to detect the expression/emotion of learners will be a challenge in future research.

  • 5.    Acknowledge

Acknowledgment addressed to Universitas Udayana as the main funder of this research that has been listed at contract No. B/78.489/UN14.4.A/PT.01.03/2022 as Penelitian Unggulan Program Studi (PUPS).

References

  • [1]    A. Budiman, Fabian, R. A. Yaputera, S. Achmad, and A. Kurniawan, “Student attendance with face recognition (LBPH or CNN): Systematic literature review,” Procedia Comput Sci, vol. 216, pp. 31–38, 2023, doi: https://doi.org/10.1016/j.procs.2022.12.108.

  • [2]    R. Febrian, B. M. Halim, M. Christina, D. Ramdhan, and A. Chowanda, “Facial expression recognition using bidirectional LSTM - CNN,” Procedia Comput Sci, vol. 216, pp. 39–47, 2023, doi: https://doi.org/10.1016/j.procs.2022.12.109.

  • [3]    P. Liang, G. Liu, Z. Xiong, H. Fan, H. Zhu, and X. Zhang, “A facial geometry based detection model for face manipulation using CNN-LSTM architecture,” Inf Sci (N Y), vol. 633, pp. 370–383, 2023, doi: https://doi.org/10.1016/j.ins.2023.03.079.

  • [4]    B. D. Kumar, H. A. Mir, M. K. Ahmed, and M. T. Siddiqui, “Exam form automation using facial recognition,” Mater Today Proc, vol. 80, pp. 2236–2240, 2023, doi: https://doi.org/10.1016/j.matpr.2021.06.190.

  • [5]    V. B. T. Shoba and I. S. Sam, “Face Recognition Using LBPH Descriptor and Convolution Neural Network,” in 2018 Second International Conference on Intelligent Computing and Control Systems (ICICCS), 2018, pp. 1439–1444. doi: 10.1109/ICCONS.2018.8663009.

  • [6]    F. Deeba, H. Memon, F. Dharejo, A. Ahmed, and A. Ghaffar, “LBPH-based Enhanced Real-Time Face Recognition,” International Journal of Advanced Computer Science and Applications, vol. 10, Jan. 2019, doi: 10.14569/IJACSA.2019.0100535.

  • [7]    M. Sarwar, A. Dey, and A. Das, Developing a LBPH-based Face Recognition System for Visually Impaired People. 2021. doi: 10.1109/CAIDA51941.2021.9425275.

  • [8]    A. B. Shetty, Bhoomika, Deeksha, J. Rebeiro, and Ramyashree, “Facial recognition using Haar cascade and LBP classifiers,” Global Transitions Proceedings, vol. 2, no. 2, pp. 330–335, 2021, doi: https://doi.org/10.1016/j.gltp.2021.08.044.

  • [9]    S. Malhotra, V. Aggarwal, H. Mangal, P. Nagrath, and R. Jain, “Comparison between attendance system implemented through haar cascade classifier and face recognition library,” IOP Conf Ser Mater Sci Eng, vol. 1022, no. 1, p. 012045, 2021, doi: 10.1088/1757-899X/1022/1/012045.

  • [10]    T. Mantoro, M. Ayu, and Suhendi, Multi-Faces Recognition Process Using Haar Cascades and Eigenface Methods. 2018. doi: 10.1109/ICMCS.2018.8525935.

133