EOG-Based Eye Movement Classification and Application on HCI Baseball Game

Electrooculography (EOG) is considered as the most stable physiological signal in the development of human–computer interface (HCI) for detecting eye-movement variations. EOG signal classification has gained more traction in recent years to overcome physical inconvenience in paralyzed patients. In this paper, a robust classification technique, such as eight directional movements is investigated by introducing a concept of buffer along with a variation of the slope to avoid misclassification effects in EOG signals. Blinking detection becomes complicated when the magnitude of the signals are considered. Hence, a correction technique is introduced to avoid misclassification for oblique eye movements. Meanwhile, a case study has been considered to apply these correction techniques to HCI baseball game to learn eye-movements.


I. INTRODUCTION
The importance of eye movement tracking along with human-computer interaction (HCI) has been investigated in this paper. This approach has remained a promising method which is used in recent years to detect and analyze eye movements. Electrooculography (EOG) is an inexpensive technique used in recent years to record eye movements [1]. EOG signal classi cation is considered as the most useful control sig-nals for human-computer interface [2]. Eight directional eye movement classi cation algorithm is an effective way to ana-lyze the aftermath effect of noise in EOG signals. However, a thorough understanding of various characteristics of eye movements leads to a better understanding of eye-movement detection algorithm.
Following types of eye movements can be detected through EOG signals. temperature, relative humidity, and brightness. Blink rate is directly associated with mental state, physical activity, or fatigue [4], [5].

E. FIXATION
Fixations are the stationary state of eyes. Visual gaze is maintained in a single location during xation state. Fixations are the events that occur between two saccades. The average xation time ranges from 100ms to 200ms [6].
In recent years, several eye tracking techniques have evolved which allow the detection and monitoring of eye movements. One of them is Infrared oculography (IR), which is generally used to quantify the difference between the amounts of infrared light re ected by the sclera and sen-sor (phototransistor) pair [7]. However, IR is not a reason-able technology to measure pursuit or saccades because of the nonlinearity problem. Many other techniques such as search eye coil [8], [9], video images [10], [11] and EOG have been proposed to track eye movements [12], [13]. EOG has been very popular due to its ease of signal acquisition approach. However, studies show that hybrid brain-computer interface utilizing hybrid signal are in prac-tice [12], [14], [15], these papers concentrate on EOG based eye movement analysis. EOG measurement is based on the potential difference between electrodes from the skin it is placed. Human eyes act like a dipole with cornea acting as positive side and retina as a negative side. When eye-balls are rotated, the inner dipoles also move consequently. These movements of eye dipoles make electrical potential slightly change around the eyes. Thus the potential difference assessing eyeball rotation can be measured. Because of these characteristics, EOG signals are considered as an appropriate approach to develop human-computer interface (HCI). It also aids in translating eye movements into human understandable commands.
EOG has become a preliminary eye movement detect-ing technique in developing HCI systems such as voice recognition [16], [17], visual information [18], gesture con-trol [19], [20], methods based on brain signals, infrared head-operated joysticks [21] and many other medical usages. Extensive research is being carried out in terms of non-medical applications such as gaming [22] [25], and browsing internet [26]. However, this paper aims to utilize EOG based classi cation in gaming applications for practical consumption. It discusses an approach to have high accuracy and low computation for an EOG-based HCI baseball game. Figure 1 gives the overview of the proposed BCI system. BCI system focuses on aspects of extracting EOG sig-nals. An EOG measuring device will be used to record the eyemovements from the subjects. A signal acquisition sys-tem is used to collect EOG signals from the devices and the processed signals are transmitted to personal devices with the aid of Bluetooth devices. Thereby, HCI computations are carried out. Classi cation algorithms are applied for VOLUME 7, 2019  eye-movement detections, and the output is represented by a graphical user interface.

A. EOG MEASURING DEVICE
An EOG Mindo device from National Chiao Tung University Brain Research Center has been used to measure EOG signals from subjects. Electro-physiological signals are measured by placing electrodes around eyes as shown in the gure 2. Electrode placed on the forehead is a reference signal. Four channels are read by placing electrodes around eyes, where Ch1 and Ch2 collect horizontal signals, and Ch3 and Ch4 col-lect vertical signals.

B. SIGNAL ACQUISITION
The proposed wireless EOG signal acquisition device was approximately 45 32 8 mm3 in size. A Bluetooth module was employed to transmit the EOG signals wire-lessly. The Bluetooth module BM0203 provided a suf cient transfer band rate (115 200 b/s) and was compliant with the computer's Bluetooth v2.0 with enhanced data rate (EDR) speci cation. Power was supplied by a lithium battery with an output voltage of 3V. A commercial 750 mA h Li-ion battery has been used to supply power to the EOG acquisition circuit, which has capacity to operate continuously for 12 hours. EOG signals are measured by the wet or dry sensors which are rstly ampli ed by the preampli er unit. The preampli er ampli es the voltage difference between the reference sig-nals and those of the EOG electrodes, while simultaneously rejecting common-mode noise (i.e., the power line noise). An instrumentation ampli er (INA2126, Texas Instruments,  Dallas, TX, USA) was used for its extremely high input impedance and high common-mode rejection ratio (CMRR) ( 90 dB) [27].
Instrumentation ampli ers have the ability to improve CMRR and amplify the EOG signals to a degree, where the minute voltage levels can also be detected. Gain of the pream-pli er unit was set to 5.5 V/V. The cutoff frequency was reg-ulated at 0.1 Hz by using a high-pass lter. Microcontroller program which is controlling preampli er and lter stage has reduced the 60 Hz noise in the EOG signals employing a mov-ing average. In addition, a 12-bit resolution ADC has been used to digitize the EOG signals. A microcontroller unit was also used to digitize the EOG signals, with a sampling rate of 256 Hz. The sync lter removed signals with frequencies higher than 62.5 Hz. After removing the noise and amplifying the EOG signals, the data was transmitted to the computer interface via a wireless module.

C. SIGNAL CLASSIFICATION ALGORITHM
EOG Classi cation algorithm is designed to reduce the overall calculation time and it also does not require signal down sampling. The structural overview of the classi ca-tion algorithm is as shown in gure 3. A software program gathers four channels transmitted from a Bluetooth device. System reduces the common mode noise caused by elec-tromyography (EMG) and environmental noise. Raw signals are obtained in horizontal and vertical form. In order to extract features from the eye-movement, raw signals need to be smoothened. Calculation amount of the signal has been reduced by introducing buffer in the classi cation phase.

1) RAW EOG SIGNAL
Electrodes are placed around the eyes to record EOG signals. During this process traces of EMG signals are found due to facial contact of electrodes. This paper intends to discuss extracting only the EOG signals. Hence, EMG signals needs to be removed from the raw signals. Equation (1) and (2) demonstrates the subtraction of channel 2 from channel 1 and channel 3 from channel 4. The signal processing is done by using these equations.
a: SIGNAL SMOOTHING Some high frequency noise still could corrupt the signal in an unexpected way. Thus, to solve this problem, a ltering process in the rmware level is introduced. A moving average method is utilized, to t the limitation of the hardware. Moving average also called rolling average, is the basic type of FIR lter in DSP domain. Moving average is most commonly used with time series data to smooth out short-term uctuations and highlight long-term trends or cycles. The choice between short-and long-term, and setting of moving average parameters depends on the requirement of application. Mathematically, the moving average is a type of convolution and similar to a low-pass lter used in signal processing. The moving average lter is optimal for a common task: reducing random noise while retaining a sharp step response.  (3).
As mentioned above, the recorded signals are easily inter-fered by 60Hz noise, especially when the acquisition circuit gets closer to the electric appliances. It has been showed in the gure 4, that the original sine wave had been contaminated by 60Hz power-line noise. After applying the moving average lter with a 5-point moving window, the moving average could be effectively removed by power-line noise, as shown in the gure 5.
Given a continuous noise signal x(t) with frequency F Hz, it is apparently that the integral within 1/F sec is equal to zero. A digital situation is demonstrated here. Equation (3) can be extended to digital form. That means the summation of all  discrete signals with one period is equal to zero as shown in equations (4) and (5).
The moving window size is decided by both sampling rate and the noise frequency as shown in equation (6).
Noise Frequency F b: BUFFER Computational expense of the system can be reduced by introducing buffer which is employed to retrieve temporary data. Computation occurs only when the buffer is full. Hence, it avoids the unnecessary computation there by increasing the ef ciency of classi er unit.
2) FEATURE EXTRACTION In order to analyze the eye-movements from EOG, meaningful features needs to be recognized and extracted. Distinguishable patterns present in saccades makes it easy to be classi ed further. Primarily, blinks and saccades needs to be segregated. Secondly, more than one eye movement needs to be identi ed based on this study.

3) CLASSIFIER
Differentiation and peak detection play an important role in the classi cation algorithm. Differentiation is used to observe the variation of the slopes which can distinguish blinking and other eye-movement ef ciently. Figure 6 demonstrates eye-movement classi cation based on magnitude variation tech-nique. However, this approach is not used to detect certain eye-movements.
Hence, signal classi cation requires a novel approach to identify blinks in a comprehensive manner, and which can also decrease the correction rate. In this paper, a slope variation technique is used to distinguish blinks from other eye-movements. Figure 7 shows the slope variation of a look-up saccade and the slope variation of a blinking. The slope variation of the blinking is apparently larger than the VOLUME 7, 2019 look-up saccade when compared with the look-up saccade in gure 7 with the special blinking #2 in gure 6. It is discovered that their magnitude is both around 1000 V but the look-up saccade has longer duration than special blinking #2. That means the slope variation of the special blinking #2 is still larger than the look-up saccade. The slope variation method increases the ef ciency to classify blinks from other eye-movements.

a: PEAK DETECTION
Peak detection is a method designed to reduce the calculation time and the number of misclassi ed cases by detecting the peak values of the vertical and horizontal signal. Classi ca-tion algorithm will nd peak values of the differentiated sig-nals. The peak value detection is utilized to identify various types of eye-movements.

b: BLINKING DETECTION AND REJECTION
There is a need to overcome misclassi cation which might adversely affect the speci c eye-movement detection. Blinks in the signal are identi ed and removed in order to avoid the interference of blinks with horizontal and vertical signals. Interference with horizontal and vertical signal will result in misclassi cation.
A novel method has been introduced to overcome misclas-si cation caused by blinks. An ef cient way of classifying eyemovement is to differentiate signals and to extract peak value of signals is shown in gure 8. Once the peak values   has been veri ed, blinks can be easily rejected based on their threshold values. The eye-movement marked beyond their threshold values after the peak values are recognized are classi ed as blink. Once the blink has been identi ed, they are rejected to extract saccades. The system searches for peak values, and then the left signal of gure 8 is decided as a blink. System does not identify center signal, hence it is marked as a saccade. The blinking rejection process is shown in gure 9.

c: PATTERN RECOGNITION
Various eye-movement detection is done by observing the peak values of the signal. Figure 10 illustrates that the peak value of the vertical signal is marked above the upper thresh-old and hence the system considers it as a look-up saccade.
Four other eye-movements identi ed are look-up-and-left, look-up-and-right, look-down-and-left, and look-down-and-right as oblique saccades. System identi es a look-up saccade when the peak of vertical signal is marked beyond the upper threshold value. Similarly system can identify a look-left saccade when it encounters horizontal signal marked beyond  its threshold value. Combination of look-up and look-left saccades forms a look-up-and-left saccade eye-movement as shown in gure 11. However, both look-up saccade and look-left saccade have to occur at the same time. A misclassi-cation is created when there is a mismatch in the occurrence of two signals. This misclassi cation can be removed by applying the exception correction.

D. GRAPHICAL USER INTERFACE DESIGN
This paper aims to present classi cation results on a HCI baseball game platform. An initial baseball game interface is shown in the gure12. A time range is set up to display the data, le and name. Once all the required information is gathered, device is paired with a Bluetooth device to stimu-late interface. Figure 13. Simulating interface is activated by pressing start button and it will guide user through different steps. This will aid us to record user reaction and to recognize various eyemovements. The total number of eye-movements occurred during this session can also be registered.

III. EXPERIMENTAL SETUP
Three aspects of experimental set up has been discussed in this paper. First experiment set up is to verify the classication working capability by considering normal scale and cues. Second experiment setup tests the capability of the classi cation by eliminating cues and using the same scale  as the rst test experiment. Third experiment is to test the classi cation functionality on a tablet by reducing original scale size to half of its size as to make it work on a tablet while considering the cues. Eye-movement is detected based on the horizontal and vertical threshold values of EOG signals. A Matlab based approach has been utilized to analyze the recorded EOG signal. The calibration interface utilized in this project can distinguish various eye-movements based on the threshold values. Figure 14 shows a simple and effective calibration interface system. Initially user needs to press ''Start calibration'' button, and the calibration will show the cue in the center of the frame. Then the up-right red dot will show up, now the user will have two seconds to move their eyes to the upright position. Similarly, experiments will be repeated for down-right position. This experiment position will be repeated for 10 times for the system to collect suf cient data to set up an appropriate threshold value.
An experimental environment is set up to mimic the day to day computer usage. Hence, a distance of 50cm is maintained between the viewer and the monitor. Look-up and look-down distances from eyes and monitor are maintained at 11cm. Look-right and look-left distance from eye to monitor is 13cm. This experiment is set up on a 22'' monitor. Figure 15 shows the experiment set up. Since the magnitude and accuracy [28] of EOG signal depends on the angular velocity, distance is transferred into the angle which is convenient to establish the relation between EOG signal   and the scale of screen. Table 1 illustrates the calculated angle of view. The above equation can be extended to digital form. That means the summation of all discrete signals with one period is equal to zero.

A. EXPERIMENT PROCEDURE WITH CUES
In day-to-day activities, eight directional saccades and x-ation are observed. Different color code is assigned for respective eye-movements as shown in gure 16. Look-up, look-down, look-right, look-left, look-right-up, look-right-down, look-left-up and look-left-down are represented by red, orange, green, yellow, blue, aqua blue, violet and navy blue respectively.

B. EXPERIMENT PROCEDURE WITHOUT CUES
This experiment is designed to simulate an intuitive technique while using the EOG application. Cues have been elimi-nated so that the user don't have to limit their eye moves in a particular direction. Process of this experiment is empty for initial 2 seconds. For the next 5 seconds the subject is     asked to move eyes in any direction. Color code represents the respective eye-movement as described in the previous section. Figure 17 shows the experimental set up without cues. Primarily, this experiment is intended to provide a natural approach to play HCI game by allowing user to have an independent eye-movement.

C. EXPERIMENT PROCEDURE WITH CUES USING SMALL SCALE
This experiment is repeated similar to the previous set up by narrowing down the scale. A challenge has been encountered while narrowing the scale, as the scale is narrowed the dis-tance between eye and the monitor is also reduced. This will cause the signal to be smaller in amplitude and it becomes dif cult to classify the signal. It will also raise misclassi -cation due to the signal direction being deviated from the expected direction. In order to use this EOG classi cation algorithm on a tablet, the scale is narrowed about half of the original size. Figure 18 shows that as we de ate the scale to 6 cm X 6.5 cm, it allows users to see the tablet from 41.7 cm distance. Now the shrinking scale will change the threshold that classi es eye-movements because the distance and the angle of view are smaller. A calibration interface has been designed to t the screen size. This has been stimulated on the PC as shown in gure 19. As shown in gure 19 each of them has three red points. First user needs to focus on the center red point, after the cue vanishes, the user will now have two seconds to make an eye-movement. The user is asked to look at the right-up red point for ve times. Each time the user is given two seconds to look at the point. Later, the user is asked to follow the similar pattern in the right-down direction. The system will acquire required information from these eye-movements.

IV. RESULTS
EOG signal is considered in this study to differentiate various eye-movements of the subjects. A classi cation technique is provided which removed 90% of blinks along with extracting required saccades. Hence, it is effective in removing blinks. Overall computational time has been reduced by eliminating down sampling of the EOG signals. This has increased ef -ciency of the classi cation system.    correct rates are slightly deteriorated from that procedure with cues. The correct rate of the number 5 has lowered signi cantly.

CUES ON SMALL SCALE DEVICE (SSD)
Result obtained by procedure with cues on a small scale device show that there is a decrease in the correct rate. Table 5 shows that the correct rates of number 2, number 4, number 6 and number 8 have increased from that of previous results. It signi es that the proposed classi cation techniques works appropriately for small scale screens. However, the correct rate of number 1, number 3, number 7, number 9 are considerably low. This classi cation can t the small scale, it can be applied on the tablet.

D. RESULTS OF APPLICATION ON HCI BASEBALL GAME
The setting up for the HCI Baseball game is as shown in gure 20. Firstly, press the ''START'' button to enter the HCI Baseball game. A translucent panel with the numbers will show up. Number 5 in the center on the panel is brighter than other numbers. Subsequently, the next number will randomly light up and it will blink. While the number is blinking, we move the eyes towards the blinking number, from the center of panel. If the blinking number is 5, eyes still stand on the center of the panel. Accuracy rate as shown in gure 21. Since every run has 10 trials, each run of the interface will show a number and the user repeats the task 10 times. A correct rate is obtained by dividing it by ten trials. The correct rate has increased and hence this EOG classi cation can be leveraged into real life scenario.

V. DISCUSSION
Experimental results have demonstrated that the proposed classi cation techniques provide high accuracy and have improved the uency of HCI game interpretation methods. Stable classi cation is obtained by conducting experiments with cues. Most of the blinks were removed during this classi cation technique and the oblique eye-movements are well classi ed with the above method. When the experiment was conducted without cues, blinks were not removed effectively due to processing time. Hence, a buffer was implemented which aided in classifying eye-movements. This system will split the signal when it encounters a blink before passing it through buffer. This will cause misclassi cation. This factor explains the decrease in correctness rate for experiment pro-cedures without cues for number 5.
The average correct rate of the result for experiment with cues in the small scale is lower than the average correct rate of the result for experiment with cues. This can be observed for the correct rate of number 1, number 3, number 7 and number 9. This circumstance will explain that the angle of view is smaller, which can make the EOG signal smaller and the EOG signal is proportionate with the angle of view. When the oblique eye-movement distance is longer from the screen, the signal of the vertical and the horizontal are smaller than the up, down, right and left eye-movements.
It is evident from gure 21 that the oblique eye-movement signal is smaller than the look-up saccade or look-right saccade. This occurrence demonstrates that the signal scale is about ten times smaller than the original signal and it is caused by electrode displacement. Channel captured for look-up saccade is clear and hence appear large. When an oblique eye-movement occur, the left eye will not directly approach the channel 2 or channel 3. Therefore, vertical and the horizontal signal of the oblique eyemovement are smaller than the up, down, right and left eyemovements. Small scale has the smaller angle of view than the normal scale, apparently the signal in small scale is smaller than normal scale. The other key point is that if there is a slight disturbance while using the tablet, this classi cation can tolerate a bit of deviation. That is because,   classi cation applies differentiation. This will shrink the magnitude of signals which makes deviation smaller. Figure 23 shows two saccades without differentiation, and the deviation is 293 micro-volt. In gure 24 we can observe two saccades with differentiation, and the deviation is 15 units. When a threshold is set by the calibration, the error probability of the two saccades without differentiation is higher than two saccades with differentiation. It aids differen-tiation to shrink the scale of the signals and this can shrink the deviation at the same time which in turn decreases the error probability.

VI. CONCLUSION
It is evident from the HCI Baseball game that the classi cation can be utilized in everyday life. Usability and simplicity of the classi cation is made ef cient due to online computation. The performance accuracy of the system has been improved by scaling down the measurement to t a tablet. The proposed method has established that by utilizing eight eye-directional movement the accuracy and performance of the system can be increased. Research conducted based on procedures without cues and small scale measurements calls for a further study in terms of improving the accuracy.
In future, we focus on developing descriptive alternatives for all directions and even smaller scale eye-movements clas-si cation and also on the implementation of a stable classi -cation on the circuit board. This EOG device can work freely like a remote controller or a joystick.