LONTAR KOMPUTER VOL. 12, NO. 1 APRIL 2021

DOI : 10.24843/LKJITI.2021.v12.i01.p06

Accredited Sinta 2 by RISTEKDIKTI Decree No. 30/E/KPT/2018

p-ISSN 2088-1541

e-ISSN 2541-5832

Electrooculogram (EOG) based Mouse Cursor Controller Using the Continuous Wavelet Transform and Statistic Features

Triadia1, Inung Wijayantoa2, Sugondo Hadiyosob3

aSchool of Electrical Engineering, Telkom University Bandung, Indonesia

1[email protected]

2[email protected]

b School of Applied Science, Telkom University Bandung, Indonesia

3[email protected] (corresponding author)

Abstract

This study design a system prototype to control a mouse cursor's movement on a computer using an electrooculogram (EOG) signal. The EOG signal generated from eye movement was processed utilizing a microcontroller with an analog to the digital conversion process, which communicates with the computer through a USB port. The signal was decomposed using continuous wavelet transform (CWT), followed by feature extraction processes using statistic calculation, and then classified using K-Nearest Neighbors (k-NN) to decide the movement and direction of the mouse cursor. The test was carried out with 110 EOG signals then separated, 0.5 as training data and 0.5 as test data with eight categories of directional movement patterns, including up, bottom, right, left, top right, top left, bottom right bottom left. The highest accuracy that can be achieved using CWT-bump and kurtosis is 100%, while the time needed to translate the eye movement to the cursor movement is 1.9792 seconds. It is hoped that the proposed system can help assistive devices, particularly for Amyotrophic Lateral Sclerosis (ALS) sufferers.

Keywords: cursor movement, CWT, EOG, statistic, k-NN.

  • 1.    Introduction

Modern technology in the health sector in monitoring and as a tool for bodily functions makes it very easy for its users. Eye-tracking technology has enabled the movement of the human eye to be used as a Human-Computer Interface (HCI) [1], [2]. The application of the HCI system based on eye movements as a human-computer interaction communication was applied to patients with Amyotrophic Lateral Sclerosis (ALS) or other diseases that experience paralysis of the hands [3]– [6]. ALS was a neurodegenerative disease of motor nerve cells that develops rapidly and is caused by damage to nerve cells in the brain [7], [8]. Patients with ALS experience paralysis of the muscles in their limbs and speaking difficulty; thus, it was difficult for ALS people to use their hands or voice to communicate with other people [7].

Apart from being used in the HCI field, human eye movement was also useful in various fields, such as healthcare, security systems, and interface design [9]–[11]. In intelligent transportation, eye movements were also useful for detecting the driver's attention level, which indicates the level of driver's drowsiness [12], [13].

On eye movement, the cornea and retina's potential produces a source of the Electrooculogram (EOG) signal. The application of EOG based control system has been commonly proposed, for example, in the control of mobile robots [14] and wheelchairs [15]–[17]. Meanwhile, computer interaction development has recently become an important issue to implement, for example, cursor control. Horizontal and vertical eye movement and flashing signals controlling the mouse cursor system by moving the direction from the EOG-based cursor [18]. Therefore, this study proposes a mouse cursor control system using EOG signals. The proposed system consists of

an EOG signal recorder, USB interface, and feature extraction and decision-making applications. The raw EOG signal was decomposed using a wavelet transform and then calculating the statistical features into a feature vector that becomes the classification algorithm's input. This system was designed to move the mouse cursor, including up, bottom, right, left, top right, top left, bottom right, and bottom left.

This paper is structured as follows, section 2 describes the design and implementation of the proposed system, including hardware design, software design, feature extraction, and classification process. Section 3 contains an explanation of implementation results followed by a discussion. The final section briefly describes the conclusions and implications of this study.

  • 2.    System Design and Implementation

The design and implementation of the system in this study were to adopt the human-computer interface (HCI) mechanism. The output of this system was a mouse cursor movement control with an EOG signal.

  • 2.1.    Hardware Design

Figure 1 shows the two components of the EOG consisting of horizontal and vertical obtained from five electrodes placed around the eye. These were attached on the edges of both eyes and also over and under the eye. The middle electrode serves as a reference. The EOGv1 and EOGv2 electrodes obtain relative corneal-retinal vertical motion of the eye, while EOGh1 and EOGh2 get a signal from the potential relative to the horizontal movement of the eye.

Figure 1. Mouse cursor controller system overview

The component for horizontal EOG signal acquisition is obtained by subtracting the left-eye electrode signal from the right eye electrode signal (EOGh = EOGh2 - EOGh1). The vertical EOG component was obtained by subtracting the signal at the eye's bottom edge from the signal at the top edge of the eye EOGv = EOGv2-EOGv1. EOGh and EOGv were notations that denote the horizontal and vertical elements of EOG.

This system consists of hardware for EOG signal acquisition and software for signal processing and decision-making, as shown in Figure 2. EOG hardware contains components for signal acquisition, consisting of an instrumentation amplifier, low pass filter (LPF), high pass filter (HPF), and level shifter. The instrumentation amplifiers amplify the electrode signal leads. The instrumentation amplifier component used in this study was INA118P with an amplification of 1000 times.

Figure 2. The components of the mouse cursor controller system


Figure 3. The schematic design of the amplifier (2-Channel EOG)


Figure 3 shows the schematic design of the 2-channel EOG amplifier used in this study. The HPF implemented in this study has a cut-off frequency of 0.05 Hz to eliminate low-frequency noise due to body movement. The value of R was obtained by applying Equation (1).

F =1πRC c2

(1)


Here the value of C = 2.2μF then the R-value was obtained R = 1.4MΩ .

Meanwhile, the LPF was designed to have a cut-off frequency of 40 Hz to reject a large amount of high-frequency noise such as muscle noise. The low pass filter was designed using the Butterworth 4th order filter method and the Sallen-key circuit type, as shown in figure 4.

Figure 4. Schematic design of low pass filter 40 Hz

To match the reading range of the ADC component, the signal was amplified by the final amplifier. Before amplification, the EOG signal has a relatively small amplitude of about 3.5 mV, so that at the end of the amplifier, 120 times gain was required. The final amplifier was designed using OP07. Then, so that the ADC can ultimately convert all EOG signals, a level shifter was designed to make all EOG signal components positive. The schematic of the amplifier and the level shifter is shown in Figure 5.

Figure 5. Schematic design of the amplifier and level shifter

  • 2.2.    Software Design

The design software developed was used to display the EOG signal's output, feature extraction, classification, and simulation of the mouse cursor. The software design with ADC reading uses the Arduino IDE. It performs serial calibration with python to perform mouse cursor direction movements with the py.mouse library data obtained from python with *.csv format.

At the classification stage, there were training data and test data. The training data was used as

the calibration data when sampling the mouse cursor motion data, while the test data with test data that has been adjusted with the calibration data was processed to determine the accuracy with the method used in this study. The classification process was shown in Figure 6.

Figure 6. Classification process


The systematic workflow was started by signal acquisition using the EOG hardware, followed by the signals' amplitude normalization. The signal was decomposed by the wavelet transform then characterized using statistical analysis including entropy, mean, kurtosis, and skewness. This feature vector was then stored as training data for each eye movement. The new input feature vector was then classified based on the vector closest to the training vector. The classification process was carried out using k-Nearest Neighbor (k-NN) with k = 3. The k-NN was chosen because it has low computational cost and effectiveness in hardware implementation.

  • 2.3.    Feature Extraction and Classification

Feature extraction in this study is used to calculate the features contained in the signal as the first step in signal classification. Feature extraction is calculated on the wavelet decomposition signal. In this study, statistical calculations were used for feature extraction. The calculated statistical parameters include:

  • 1.    Mean

For a N number data of a set X, the mean (μ) can be calculated using (2)

N

μ =—∑X N ∑

(2)


  • 2.    Entropy

Entropy is used to measure the irregularity of signal distribution (p). The entropy calculation is shown in (3).

N-1

entropy = -∑ p (i) log2 (p)                        (3)

i=0

  • 3.    Skewness

Skewness is the symmetry value of a set X , and it is calculated using (4).

skewness =


ZNjX1-XiZN-


σ


3


(4)


Here, the mean, the σ and N is the standard deviation, and the number of data, respectively.

4.


Kurtosis

Kurtosis calculates the relative sharpness of a signal's distribution curve, which calculates using (5).


kurtosis =


∑ Z, (Xi - X )4,Zn


σ4


(5)


These parameters then become the feature vector as input fork-NN to be classified. In this study, the method of measuring the distance in k-NN is the Euclidean method.

  • 3.    Result and Discussion

This section discusses the results of testing and analysis of the system that has been implemented. This test aims to determine system performance. Testing was done during eye movement to control mouse movement. The test was carried out on seven individuals with normal vision by moving the sun horizontally (left and right movement) and vertically (up and down movement). Figure 7 shows an example of an EOG signal when the eyeball is left and right, respectively. This signal is then decomposed with a wavelet, and its statistical characteristics are calculated.

The test scenario consists of four procedures. The first procedure was equipment preparation, which checks and verifies the connection of the equipment used. This procedure was done to ensure all electrodes' positions are confirmed and ready to be used. The second procedure was user preparation, which focused on the electrodes' placement in the participant's face. Before the electrodes were placed, the face surface must be cleaned using a gel cleanser. The vertical electrodes were placed above the right eyebrow, and the lower lid, with distance, is set for about 1 cm and 1.5 cm, respectively. The horizontal electrodes were placed in the outer canthi for about 1.5 cm on each side. The reference electrode was placed on the forehead.

Figure 7. Example of an EOG signal (left and right eyeball)

The third procedure was system calibration. It was started by the calculation of eye blinks and movement. Thus it can be used as the system's threshold. The threshold for each participant was calculated by calculating the amplitude of their various eye movements. The eye movement was measured by giving the participants visual stimulation using a video showing a moving square object. The square moved to five different locations and stayed for five seconds on each location. The last procedure was exiting the calibration process. The calibration process was ended when the participants make a spontaneous blink using the right eye.

In EOG signal processing, there were three types of CWT wavelets used in this study: morse, amor, and bump. The feature extraction method was done using the statistical features of mean, entropy, skewness, and kurtosis. Before the testing phase, a system training stage using Euclidean distance with a value of k = 3 was performed using the training data. After that, the testing data was fed to the system to be classified.

Figure 8. Effect of mother wavelet and statistical features on system accuracy

Figure 8 shows that the effect of wavelet types and statistical features on the accuracy of the generation. The result shows that there is no significant difference between the use of statistical measurement and entropy. However, the signal sharpness analysis using kurtosis in the bump CWT can provide the best performance, which was 100%. Since the EOG signal was not symmetrical, signal analysis using skewness could not give a good result. Therefore, the use of skewness achieves the lowest accuracy of 69%.

Furthermore, the computation time for each test scenario was shown in Figure 9. The kurtosis feature takes longer than other features (1.9792 seconds) but provides the highest accuracy. The difference in processing time was not significant, so that if this system is applied, the characteristic of kurtosis was most suitable to be used by considering the accuracy.

Figure 9. The computation time of each scenario

The mouse control system proposed in this study is expected to help people with disabilities when they want to operate a computer with simple commands. A control system using EOG signals may be the last alternative if the hands and feet are also disabled. This proposed study can complement the previous study by Rusydi et al. [19], where muscle signals and EOG can be utilized for the control system.

  • 4.    Conclusion

In this study, a mouse control system using EOG signals has been successfully implemented. The EOG signal was decomposed using a wavelet transform, and then the statistical features

were calculated, including entropy, mean, kurtosis, and skewness. K-Nearest Neighbor was used to classifying the mouse's moving, including up, top right, top left bottom, bottom right, bottom left, right, and left. From the proposed system's test results, the highest accuracy was 100%, obtained using the statistical features of kurtosis and wavelet bump with a computation time of 1.9792 seconds. The proposed system is expected to be used by people with disabilities to operate computers with simple commands. In future studies, a user interface similar to a keyboard compatible with the operating system will be developed to write text. Another important issue is that this system requires a faster processing time to run in real-time.

References

  • [1]    S. Chandra, G. Sharma, S. Malhotra, D. Jha, and A. P. Mittal, "Eye tracking based human computer interaction: Applications and their uses," in Proceedings - 2015 International Conference on Man and Machine Interfacing, MAMI 2015, 2016, no. December, pp. 1–5, doi: 10.1109/MAMI.2015.7456615.

  • [2]    X. Zhang, X. Liu, S. M. Yuan, and S. F. Lin, "Eye Tracking Based Control System for Natural Human-Computer Interaction," Computational Intelligence and Neuroscience, vol. 2017, pp. 1–9, 2017, doi: 10.1155/2017/5739301.

  • [3]    D. Y. Kim, C. H. Han, and C. H. Im, "Development of an electrooculogram-based humancomputer interface using involuntary eye movement by spatially rotating sound for communication of locked-in patients," Scientific Reports, vol. 8, no. 1, pp. 1–10, 2018, doi: 10.1038/s41598-018-27865-5.

  • [4]    C.-Y. Su and J.-J. Wong, "Connecting with Dysphonia: Human-Computer Interface for Amyotrophic Lateral Sclerosis Patients," 2011, pp. 453–457.

  • [5]    H. Ka Hou and S. K.G., "Low-Cost Wireless Electrooculography Speller," in 2018 IEEE International Conference on Systems, Man, and Cybernetics (SMC), Oct. 2018, pp. 123– 128, doi: 10.1109/SMC.2018.00032.

  • [6]    G. Teng, Y. He, H. Zhao, D. Liu, J. Xiao, and S. Ramkumar, "Design And Development Of Human Computer Interface Using Electrooculogram With Deep Learning," Artificial Intelligence in Medicine, vol. 102, p. 101765, Jan. 2020, doi: 10.1016/j.artmed.2019.101765.

  • [7]    O. Hardiman et al., "Amyotrophic lateral sclerosis," Nature Reviews Disease Primers, vol. 3, no. 1, p. 17071, Dec. 2017, doi: 10.1038/nrdp.2017.71.

  • [8]    E. Zucchi et al., "Neurofilaments in motor neuron disorders: towards promising diagnostic and prognostic biomarkers," Molecular Neurodegeneration, vol. 15, no. 1, p. 58, Dec. 2020, doi: 10.1186/s13024-020-00406-3.

  • [9]    D. Yuan et al., "A closed-loop electrical stimulation system triggered by EOG for acupuncture therapy," Systems Science & Control Engineering, vol. 8, no. 1, pp. 128–140, 2020, doi: 10.1080/21642583.2020.1733130.

  • [10]    A. Bissoli, D. Lavino-Junior, M. Sime, L. Encarnação, and T. Bastos-Filho, "A human– machine interface based on eye tracking for controlling and monitoring a smart home using the internet of things," Sensors (Switzerland), vol. 19, no. 4, pp. 1–26, 2019, doi:

10.3390/s19040859.

  • [11]    C.-I. Wu, "HCI and Eye Tracking Technology for Learning Effect," Procedia - Social and Behavioral Sciences, vol. 64, pp. 626–632, Nov. 2012, doi: 10.1016/j.sbspro.2012.11.073.

  • [12]    A. Sahayadhas, K. Sundaraj, and M. Murugappan, "Detecting Driver Drowsiness Based on Sensors: A Review," Sensors, vol. 12, no. 12, pp. 16937–16953, Dec. 2012, doi: 10.3390/s121216937.

  • [13]    J. Xu, J. Min, and J. Hu, "Real-time eye tracking for the assessment of driver fatigue," Healthcare Technology Letters, vol. 5, no. 2, pp. 54–58, 2018, doi: 10.1049/htl.2017.0020.

  • [14]    W. S. Sanjaya, D. Anggraeni, R. Multajam, M. N. Subkhi, and I. Muttaqien, "Design and Experiment of Electrooculogram (EOG) System and Its Application to Control Mobile Robot," Journal of Physics: Conference Series, vol. 180, pp. 1–8, 2017, doi: 10.1088/1742

6596/755/1/011001.

  • [15]    R. B. Navarro, L. B. Vázquez, and E. L. Guillén, EOG-based wheelchair control, Second Edition Elsevier B.V., 2018.

  • [16]    N. Borkar, T. Dongare, P. Chahande, J. Bonsod, and A. B. Jirapure, "Microcontroller Based EOG and Accelerometer Guide Wheelchair," International Research Journal of Engineering and Technology (IRJET), vol. 5, no. 3, pp. 3803–3807, 2018.

  • [17]    W. Xu, N. Chen, X. Han, and J. Sun, "Research on wheelchair robot control system based on EOG," AIP Conference Proceedings, vol. 1955, no. April, pp. 1–5, 2018, doi:

10.1063/1.5033815.

  • [18]    A. U. Kabir, F. Bin Shahin, and M. Kafiul Islam, "Design and Implementation of an EOGbased Mouse Cursor Control for Application in Human-Computer Interaction," Journal of Physics Conference Series, vol.  1487, no. 1, pp.  1–6, 2020, doi: 10.1088/1742

6596/1487/1/012043.

  • [19]    M. I. Rusydi, I. Aryeni, Joefrinaldo, Z. Romadhon, and A. Rusydi, "Robot mobile control based on three EMG signals using an artificial neural network," IOP Conference Series: Materials Science and Engineering, vol. 602, no. 1, pp. 1–11, 2019, doi: 10.1088/1757-899X/602/1/012028.

61