Decoding Eye Blink and Related EEG Activity in Realistic Working Environments

Accurately evaluating cognitive load during work-related tasks in complex real-world environments is challenging, leading researchers to investigate the use of eye blinking as a fundamental pacing mechanism for segmenting EEG data and understanding the neural mechanisms associated with cognitive workload. Yet, little is known about the temporal dynamics of eye blinks and related visual processing in relation to the representation of task-specific information. Therefore, we analyzed EEG responses from two experiments involving simulated driving (re-active and pro-active) with three levels of task load for each, as well as operating a steam engine (active vs. passive), to decode the temporal dynamics of eye blink activity and the subsequent neural activity that follows blinking. As a result, we successfully decoded the binary representation of difficulty levels for pro-active driving using multivariate pattern analysis. However, the decoding level varied for different re-active driving conditions, which could be attributed to the required level of alertness. Furthermore, our study revealed that it was possible to decode both driving types as well as steam engine operating conditions, with the most significant decoding activity observed approximately 200 ms after a blink. Additionally, our findings suggest that eye blinks have considerable potential for decoding various cognitive states that may not be discernible through neural activity, particularly near the peak of the blink. The findings demonstrate the potential of blink-related measures alongside EEG data to decode cognitive states during complex tasks, with implications for improving evaluations of cognitive and behavioral states during tasks, such as driving and operating machinery.

Abstract-Accurately evaluating cognitive load during work-related tasks in complex real-world environments is challenging, leading researchers to investigate the use of eye blinking as a fundamental pacing mechanism for segmenting EEG data and understanding the neural mechanisms associated with cognitive workload.Yet, little is known about the temporal dynamics of eye blinks and related visual processing in relation to the representation of task-specific information.Therefore, we analyzed EEG responses from two experiments involving simulated driving (re-active and pro-active) with three levels of task load for each, as well as operating a steam engine (active vs. passive), to decode the temporal dynamics of eye blink activity and the subsequent neural activity that follows blinking.As a result, we successfully decoded the binary representation of difficulty levels for pro-active driving using multivariate pattern analysis.However, the decoding level varied for different re-active driving conditions, which could be attributed to the required level of alertness.Furthermore, our study revealed that it was possible to decode both driving types as well as steam engine operating conditions, with the most significant decoding activity observed approximately 200 ms after a blink.Additionally, our findings suggest that eye blinks have considerable potential for decoding various cognitive states that may not be discernible through neural activity, particularly near the peak of the blink.The findings demonstrate the potential of blink-related measures alongside EEG data to decode cognitive states during complex tasks, with implications for improving evaluations of cognitive and behavioral states during tasks, such as driving and operating machinery.

I. INTRODUCTION
T HE assessment of cognitive workload is essential in stressful working environments like car driving [1], [2], operating machinery [3], and other applications [4], [5] to detect dangerous situations.However, methods like individual questionnaires and qualitative observations are subjective and discrete in time, necessitating a continuous and objective estimation of cognitive workload.Therefore, objective quantification of mental processes and states is crucial, and less intrusive methods like electroencephalography (EEG) or eye tracking can deliver a real-time and objective estimation of cognitive workload.EEG is a widely used neuroimaging technique that measures the electrical activity of the brain, offering several advantages including superior temporal resolution, noninvasiveness, affordability, portability, and the ability to capture activity from critical brain regions [6], [7].These factors make EEG an indispensable tool for investigating various cognitive and neural processes.However, a difficulty associated with EEG recordings is the requirement for event markers that signify the timing of noteworthy events.However, adding event markers based on stimulus presentation can be challenging in naturalistic settings, where visual inputs to participants continuously change.Recently, eye blinks have been proposed as valuable event markers to indicate cognitive load in real-life situations where a task is performed consistently without interruption [8], [9], [10], [11].People generate these blinks spontaneously, and such blinks can be easily detected with EEG without any extra devices, offering a non-invasive and straightforward approach to identifying notable events.
Research exploring the brain activity associated with eye blinking is essential to authenticate using eye blinks as event markers.Several studies have demonstrated that spontaneous eye blinks indicate cognitive load, often when processing intricate visual scenes [3], [8].After blinking, the brain experiences an upsurge of visual information when the individuals open their eyes, leading to visual processing-related brain activity.However, previous research linking eye blinks to cognitive load has primarily concentrated on blink features such as eye movements and blink rate variability [12], [13], [14], with less emphasis on brain activity associated with eye blinks.To the best of our knowledge, only a handful of studies have investigated the neuronal processes connected to eye blink events [3], [8], [15], and none have yet investigated the neural and blink patterns involved in decoding cognitive states that precede and follow eye blinks.
In recent research, Multivariate Pattern Analysis (MVPA) has gained attention as a technique for deciphering neural processes associated with target events in a time-resolved manner.The MVPA exhibits high sensitivity to subtle changes in neural activity patterns that are difficult to detect through traditional univariate approaches [16], [17].The primary method used to assess the temporal characteristics of visual information perception is the analysis of event-related potentials (ERPs) [18].However, the limitations of univariate ERPs are their restricted sensitivity to local neural processes and the inability to capture the contributions of multiple neural processes that occur in distributed brain regions involved in visual processing.
The multivariate decoding approach has significantly broadened the utility of ERPs beyond traditional univariate evaluations of condition differences [19], [20].The MVPA leverages voltage topographies that reflect distributed neural activities, enabling the discovery of discriminability between experimental conditions that is not achievable with the univariate ERP method.This decodable information in neural patterns has been utilized to measure differences in neural representations [21].Thus, the temporal profile of decoder performance in EEG data can provide valuable insights into the formation and evolution of neural representations associated with a particular stimulus or condition over time [22].For example, Bo et al., [19] identified affect-specific neural representations forming in the occipital and temporal cortices within 200-300 ms following picture onset.These representations persisted for two seconds, suggesting that they are organized over time in a way that depends on the emotional valence of the stimulus and are sustained by repeated neural interactions across various brain regions.Another study by Cichy et al., [23] utilized MVPA to investigate the representation of visual stimuli in the early visual cortex.The results demonstrated that MVPA could accurately decode the category of visual stimuli based on the patterns of neural activity in the early visual cortex.
Supervised machine learning algorithms, specifically L2regularized linear logistic regression [24], [25], have been shown to deliver outstanding results in the analysis of EEG data [26].The L2-norm regularization approach efficiently singles out a restricted set of relevant features from a feature space with many dimensions [27].Recently, the integration of these machine learning-based techniques with neuroimaging tools such as EEG, Magnetoencephalography (MEG), or functional Magnetic Resonance Imaging has gained significant traction in cognitive neuroscience research [28], [29].Such integration holds the potential to provide a more comprehensive understanding of the relationship between neural activity and cognitive processes.
The ability of the human brain to process and encode visual information has been extensively researched in cognitive neuroscience.However, one aspect that has received less attention is the effect of blinks on forming neural representations.In this study, we aimed to investigate the timing of neural representations after blinking and explore the possibility of decoding blink patterns from EEG components.Additionally, we examined whether different conditions, which vary in visual information, exhibited systematic timing responses in forming neural representations.To accomplish this, we analyzed EEG responses from two different experiments.One experiment comprised a simulated re-active and pro-active driving task, and the other experiment consisted of two operating conditions of the steam engine (active and passive).We investigated eye blinking and related EEG activities at three difficulty levels (low, middle, and high) for both re-active and pro-active driving, as well as active and passive operating conditions of the steam engine.We hypothesized that visual processing generated from different conditions following eye blinks could be decoded separately, exhibiting consistent temporal patterns of neural representations across these conditions.This could have important implications for understanding the neural basis of visual perception and decoding real task scenarios.

A. Participants
We conducted an analysis of the data collected from two experimental paradigms utilized in our previously published studies [1], [2], [3].The first experiment involved re-active and pro-active driving.A total of 32 participants (16 female and 16 male) were recruited for each of the re-active and pro-active driving experiments, and three participants from each experiment were excluded due to poor data quality resulting in a low number of epochs.The remaining participants were between the ages of 20 and 70 with an average age of 44 and a standard deviation of 20.They were experienced drivers, using a car at least twice a week for the past three years.
The second experiment consisted of two types of steam engine operation: active and passive.A total of 18 participants (comprising 16 males and 2 females) were selected via online announcements.However, four participants had to be excluded from the analysis due to faulty trigger markers in the EEG data, making the segmentation of conditions unfeasible.The remaining participants were between the ages of 22 and 35, with an average age of 27.29 and a standard deviation of 4.45.
All potential participants were screened for eligibility based on various criteria, including the absence of neurological or psychiatric disorders before participation.All participants provided informed written consent and received compensation of 10€ per hour.The study was approved by the local ethics committee of the Leibniz Research Centre for Working Environment and Human Factors and was conducted following the Declaration of Helsinki.

B. Procedure and Stimuli
The study employed two experimental designs, treating each experiment as separate entities.The first experiment focused on driving with two modes: pro-active and re-active driving, each with three difficulty levels [1], [2].Data for this experiment were collected using a static driving simulator (ST Sim; ST Software B.V. Groningen, Netherlands).In the re-active driving condition, participants drove on a straight two-lane road while a sinusoidal lateral force was applied to simulate crosswinds.The force comprised eight superimposed and phase-delayed sine waves with varying frequencies.During the pro-active task, the participants were required to drive on the road with a single lane and varying radii of curves.The task difficulty was manipulated by varying the amplitude of the crosswind and the radii of the curves, with three task load levels (low, middle, high) randomly presented in segments of a 2-minute duration.Participants were instructed to maintain their driving accuracy.Short transfer intervals were introduced before each task load segment to prevent sudden shifts.The experimental design consisted of 10 triplets of 3 task load levels, with the first triplet being a practice period.The entire experimental block lasted for 54 minutes without interruption.
The second experiment focused on the operation of a steam engine and involved two conditions: active and passive.Data for this experiment were collected in a seminar room that was furnished with a miniaturized steam machine having a single engine [3].The steam engine was powered by boiling water and regulating the steam flow using a valve and an adjustable Bunsen burner.A Raspberry Pi 2B was placed on top of the engine to generate trigger signals for EEG data and provide visual stimuli to the participant.USB thumb drives were used to control the sequence of visual stimuli.The participants were given two tasks.In the first task, they had to operate the steam engine to reach a specific pressure level by regulating the heat supply, incoming cooling water quantity, and pressure valve.They then recorded various temperatures and calculated the temperature difference between the incoming and outgoing cooling water.In the second task, they passively observed the system and marked the water tank temperature when the Raspberry Pi diode lit up three times at quasi-random intervals.The passive task followed the active task, and this procedure repeated three active and three passive tasks, with each task block lasting seven minutes, resulting in 42 minutes of experimentation time.

C. EEG Data Acquisition and Processing
1) Data Acquisition: During the driving task, 64 scalp electrodes were used to record EEG data (Biosemi active system, Active two, BioSemi, NL).The electrode placement was based on the International 10-10 system, along with two extra electrodes positioned on the mastoids, specifically on the left and right sides.The system employed a 2-wire active electrode approach that followed the Common Mode Sensing and Driven Right Leg (CMS/DRL) principle.The data were sampled at a frequency of 2048 Hz and a bandwidth of DC-140 Hz while maintaining the electrode impedance below 10 kΩ.
For the steam engine experiment, EEG signals were recorded using a 10-20 system montage (Brain Products GmbH, Gilching, GER) with 30 active electrodes.Electrodes were placed into electrode holders within a tight flexible cap and filled with electrolyte gel until an impedance of 10 kΩ was achieved.Cable management was carefully implemented to prevent motion artifacts.The recordings were made with an online reference at FCz, and the ground electrode was placed at AFz.The EEG signals were recorded with a mobile amplifier at the back of the participant's head and stored offline on a micro-SD card inserted into the amplifier.The data was transferred to a BrainVision Recorder laptop and converted into EEGLab readable files using the LiveAmp File converter software.The recordings were made at a sampling frequency of 500 Hz with a bit depth of 24 bits.
2) Preprocessing: The data analyses were performed through custom EEGLAB scripts using MATLAB [30].To eliminate environmental and muscular artifacts, the raw EEG data were filtered using finite impulse response (FIR) filters (eegfiltnew), with a high-pass filter at 0.1 Hz and low-pass filters at 16 Hz for driving and steam engine datasets.The clean_artifacts function was applied to identify and flag bad channels with default parameters (flatline = 0.5 s, burst = 5, line noise = 4, correlation = 0.8, and window = 0.25) before re-referencing the data to the common average.Subsequently, a high-pass zero-phase Hamming window FIR filter with a cutoff frequency of 1.5 Hz was applied to the data.The filtered data were further subjected to decomposition into statistically independent components (ICs) using AMICA [31].Finally, the obtained ICs were copied onto the average referenced data, representing the data prior to the high-pass filtering step.
Afterward, the IClabel algorithm was applied to the decomposed signal to automatically classify components and remove any ICs that were not brain-related.This included any components with < 30% brain and > 30% muscle, eye, channel, heart, and other classification probabilities.However, the occipital activity for steam engine data could be impacted by the subject's voluntary movements during the task, potentially resulting in a merging of the muscular and visual components.Thus, the cutoff frequency was set to 16 Hz, and ICs with > 30% classified as muscle were excluded to eliminate most muscular artifacts.In pursuit of consistency during data preprocessing, the driving data was also processed in a manner analogous to the steam engine data.
3) Blink Detection and Data Segmentation: The preprocessing stage at hand was concerned with identifying eye blink events and eliminating eye artifacts from the brain activity signal.It involved selecting a continuous "blink" signal based on the best eye-blink IC.Numerous methodologies exist to detect blinks from EEG [8], [9], [10], but this study focused on using the EEG BLINKER tool to extract blinks [32].BLINKER detects intervals of potential blinks in the EEG signal when the signal exceeds the overall signal mean by more than 1.5 standard deviations.The tool only considers possible blinks that last for more than 50 ms and are at least 50 ms apart.The algorithm identifies blinks by identifying landmarks such as the highest peak and zero crossings, then computing linear fits for the middle 80% of the upward and downward strokes.The R2 value of these fits determines how closely the potential blink matches a typical blink.Additionally, the blink amplitude ratio is calculated by dividing the average signal amplitude between the left and right zero crossings by the average amplitude of the positive signal.Blinks with a ratio outside the range  are excluded from the final computation.
The BLINKER algorithm selects the best blink IC based on a figure-of-merit computed from the difference between the means of frontal and rear hemisphere distributions of each component.However, this may not always select the most relevant eye blink IC, as it sometimes fails to identify eye blinks from saccade-related ICs.To address this issue, we modified the BLINKER algorithm to identify the optimal blink IC by calculating the Pearson correlation between the left and right anterior channels of eye ICs classified by the IClabel algorithm [33].The IC exhibiting a high positive correlation was identified as the eye-blink IC, while the IC with a high negative correlation was identified as the saccade IC.The selected eye blink IC was then inserted into the BLINKER tool with default parameters to identify blinks and insert corresponding event markers into the EEG.event data structure to segment the blink (continuous blink IC time series) and brain signal.Prior to this, the data were down-sampled to 256 Hz for driving data and 250 Hz for steam engine data.Epochs were then extracted from −500 ms to 1200 ms relative to the peak of the eye blink.These epochs were processed using an automatic epoch rejection EEGLAB function called "pop_autorej".This function excluded epochs with fluctuations that exceeded an absolute threshold value of 500 µV and a standard deviation threshold of 5.The exclusion process was iterative, with a maximum of 10% of epochs being rejected in each iteration.

D. Multivariate Pattern Analysis
The MVPA is an advanced technique that determines optimal topographical weights of electrophysiological signals, distinguishing perceptual states in a predefined temporal window, and identifies critical discriminative features and their spatial distributions, providing valuable insight into underlying neural mechanisms.The present study employed MVPA by feeding 50 ms integrated time windows, each consisting of approximately 13-time points, derived from signals obtained from all electrodes for the brain signal and the eye blink IC for the blink signal.
This study utilized a linear logistic regression classifier with a C parameter of 1.0 and L2 regularization from LIBLINEAR [25] to determine the optimal projections of brain and blink signals to discriminate between various conditions at a specific time.This allowed the assessment of how and when visual information is processed by the brain and the eye.The brain processes visual information from the eyes, while the blink signal provides information about the state of the eye (open or closed).To prevent biased classification outcomes, the datasets underwent rigorous balancing procedures whereby the majority class was downsampled to align with the size of the minority class.The average number of balanced epochs generated for re-active and pro-active driving data is 489, while for steam engine data is 255.For each participant, we conducted feature extraction by generating two feature vectors, one for each class or condition.These feature vectors were created using the raw potential measured across all electrodes or blink data and time points within a window of the past 50 ms (e.g., brain, 13 × 64; blink 13 × 1).The performance of the classifier was assessed by means of a Monte Carlo cross-validation (MCCV) with five repetitions, where the entire dataset was randomly partitioned into a training set (90% of the trials) and a test set (10% of the trials) in each repetition.During each repetition, the feature vector comprising raw potential values of the training set was subjected to normalization through min-max scaling within trials.To ensure consistency in the normalization process, the testing data was normalized using parameters (i.e., maximum and minimum values) estimated solely from the training set.The weight values obtained from the classifier were transformed into absolute values, which were then visualized as a topographical map.This map served the purpose of evaluating the contribution of each channel to the accuracy of decoding between different conditions.
The focus of our investigation lies within the domain of blink-related EEG data, a topic presented in this study.However, a lingering question remains: Can random EEG epochs, as a possible method for segmenting data from real-world environments without associations with events such as blinking, be decoded and interpreted in a comparable fashion?To address this, we performed MVPA on EEG epochs that were randomly selected to match the number of epochs used in the blink-related EEG dataset.The main goal was to determine whether these arbitrarily chosen epochs could be successfully decoded or capture specific neural events, and whether the outcomes would deviate significantly from chance-level performance.The results are provided in the supplement.

E. Temporal Generalization
In this study, we also used the temporal generalization matrix approach to evaluate the temporal stability of brain patterns [34].Each matrix is associated with a set of weights that facilitate optimal discrimination for a particular condition at a specific time point.For instance, decoding accuracy for a given point on the plane (t x , t y ) indicates the classifier's ability to discriminate at time point t x , which was trained at time point t y .To accomplish this, a computational model was trained on a subset of trials within a specific temporal interval, and its ability to accurately distinguish between conditions was tested using the remaining trials across the remaining intervals.This process was repeated five times for each temporal interval to achieve the final decoding accuracy.

F. Statistical Analysis
The present study employed global field power (GFP) as a quantitative measure of electric field strength [35].Statistical analyses were conducted to determine the significance of the GFP signal deviation from the pre-blink interval [−300,−100] for each experimental condition separately using paired t-tests.To control for false positive results, the resulting p-values were corrected using false discovery rate (FDR) correction across the time course [36].To assess the significance of time points in decoding, paired t-tests were conducted on the distribution of decoding performance across participants.Decoding performance was calculated using both real and shuffled labeled data.The statistical significance of the results was adjusted for multiple comparisons using the FDR (p < 0.05).
To examine the statistical significance of temporal generalization, an extreme pixel-based permutation test was employed.This test is more rigorous than a cluster-based permutation test and can detect smaller clusters [37].Our hypothesis assumed no significant differences between real and shuffled temporal generalization matrices.A null hypothesis map was created by shuffling subjects among groups, computing the mean for each new group, and subtracting one map from another.The minimum and maximum values were extracted from this new graph, and the process was repeated 1000 times to generate a bi-modal distribution.Values above 0.025 and below 0.975 were considered statistically insignificant, while values outside this range were deemed significant.This approach enabled the identification of statistically significant differences in temporal generalization.Finally, a Spearman correlation coefficient was used to assess the relationship between eye blink properties and high prediction accuracies.

III. RESULTS
The results of the GFP analysis for both re-active and proactive driving levels, as well as steam engine operating conditions, are shown in Fig. 1.In both experiments, all conditions elicited a significant electrophysiological response compared to the pre-blink period.This response commenced at the maximum blink point (time 0 ms), corresponding to the first potential peak, and continued for over 300 ms.These findings suggest that a notable amount of brain activity occurs immediately after a blink, which could potentially indicate a difference in cognitive processing between the two conditions during a similar timeframe.
In contrast to the conventional approach of analyzing human electrophysiological data by averaging across trials and subjects, we utilized MVPA to investigate whether the electrophysiological activity of individual trials contains discriminative information across different experimental conditions.To this end, we trained classifiers for one-versus-one condition discrimination at each time interval using randomly selected 90% of the trials for training and the remaining 10% for testing.The results presented in Fig. 2(a) demonstrate that levels of pro-active driving (low vs. high, low vs. middle, and middle vs. high) could be accurately distinguished above the chance level at temporally adjacent points following the blink maximum, specifically at 188 ms, 215 ms, and 234 ms, respectively.
The decoding performance analysis revealed that the brain processes high-level visual information, which was supported by the contribution of various electrodes to the high decoding performance, as demonstrated in Fig. 2(a).The classifier weight maps illustrated that the neural activities responsible for the classifier performance were predominantly located in the parieto-occipital channels.However, when analyzing re-active driving, only the comparison between low and high levels could be decoded significantly above the chance level at 188 ms after the blink maximum, with the contribution of occipital channels indicated by the weight map.In contrast, the other comparisons between low vs. middle and middle vs. high levels could not be reliably decoded.
On the other hand, when examining EEG data unrelated to eye-blinks for the binary representation of difficulty levels in both the pro-active and re-active, as well as the steam engine datasets, we observed results that did not significantly deviate from chance levels (see supplemental material, Fig. S1).Essentially, the random EEG epochs exhibited a noteworthy struggle to decoding, thereby failing to reveal any meaningful or discernible patterns.Interestingly, blink data could provide sufficient information for discriminating between different conditions above the chance level.Fig. 3(a) demonstrates that blink data can accurately decode most conditions around the blink maximum, from eye closure to opening.This finding suggests that blink data contains essential information and can complement EEG measures when evaluating cognitive states.Notably, it appears that there is potential to distinguish between low and middle, as well as middle and high re-active driving conditions with an accuracy above chance levels.However, its significance has been somewhat constrained by multiple corrections.This distinction was achieved using blink data, while EEG data, on the other hand, demonstrated similarity to chance levels.Therefore, incorporating blinks and related neural activity into cognitive and behavioral assessments could have immense potential.
A temporal generalization analysis was also performed to investigate the temporal dynamics and similarity of pro-and re-active level representations.A classifier trained at a specific time point was used to test all other time points, resulting in a two-dimensional matrix that illustrated generalized decoding profiles.The results showed a consistent processing pattern, as indicated by the narrow main diagonal decoding pattern, particularly in pro-active levels for the EEG data (Fig. 2(b)) and pro-re-active level representations for the blink data (Fig. 3(b)).This pattern revealed that classifiers trained at a particular time point (t) only generalized effectively to adjacent time points.The lack of off-diagonal decoding patterns is critical for comprehending the dynamics of neuronal data coding that occurs following eye blinks.The main diagonal cluster for the EEG data roughly starts at the blink maximum (0 ms) and lasts approximately 400 ms, while for the blink data, it commences before the blink maximum (∼ −200 ms) and lasts about 400 ms.These findings suggest that certain neural processes or cognitive operations occur at specific intervals following eye blinks, as demonstrated by the observed decoding pattern.
Moreover, the EEG and blink data revealed significant decoding of overall pro-vs.re-active driving data, including all difficulty levels, as well as active vs. passive operators, as demonstrated in Fig. 4(a) and (b), respectively.Notably, the MVPA results showed highly accurate decoding for pro-vs.re-active driving, with a peak at 172 ms after the blink maximum, where the classifier accurately predicted (>80%) whether the trials were for a re-or pro-active driving condition.We also decoded the multivariate EEG patterns evoked by the steam engine operating conditions and found that against active operators, the high-performance time of above-chance decoding for passive Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.operators was at 196 ms.Interestingly, the weight maps showed an occipital and parieto-occipital lateralized distribution at the decoding peak following the blink maximum for the driving and steam data, respectively.
Additionally, the blink data demonstrated significantly high decoding performance prior to and following the blink maximum, specifically during the closing and opening of the eye, thus confirming the reliability of the blink data as a complementary tool alongside blink-related EEG data.The temporal generalization matrices indicated similar patterns to those previously discussed, shedding light on the possibility of discriminating conditions at some time point based on the maximally discriminative weights at another time point.Fascinatingly, during driving and steam engine operating conditions, the temporal generalization profile mostly clustered around the diagonal, suggesting that distinct, sequential neural processes in the occipital area, as depicted in the weight maps, play a crucial role in encoding information following blinks.
To investigate the potential of using eye blinks as a proxy for decoding cognitive states, we analyzed the variance of eyelid closure and opening waveforms for all trials in two experimental conditions: pro-active and re-active driving (Fig. 5(a)) and active and passive operating conditions (Fig. 5(b)).These waveforms refer to the eyelid movement over time, offering a more comprehensive comprehension of the various patterns and fluctuations in eyelid movement.The results demonstrate that during eye closures and openings, the pro-active driving and active operating conditions exhibit a higher variance compared to the re-active driving and passive operating conditions, respectively, as illustrated in the top portion of Fig. 5(a) and (b).
To explore the relationship between the variance difference in both driving conditions (pro-active -re-active) and both steam engine conditions (active -passive) and the decoding performance during eye closing and opening, we conducted a Spearman's rank correlation analysis.The results revealed a strong positive correlation between the variance difference and high decoding performance.Specifically, the correlation was higher for eye closing (r = 0.38, p < 0.05 for driving; r = 0.81, p < 0.001 for steam compared to blink opening (r = 0.17, p = 0.37 for driving; r = 0.58, p < 0.05 for steam engine).These findings suggest that the variance observed in eye closing can serve as a dependable and consistent indicator of cognitive states during both driving and operating a steam engine.

IV. DISCUSSION
The present study investigated the neural and blink behavioral correlates associated with cognitive states during pro-active and re-active driving, as well as steam engine operating conditions.The study results indicated that both re-active and pro-active driving levels, as well as steam engine operating conditions, produced significant EEG activity represented by GFP following a blink.This increase in EEG activity was most prominent up to 400 ms after the blink compared to the pre-blink period.This variation in EEG activity may be attributed to the high cognitive activity that occurs after a blink.Previous studies have examined the relationship between blinking and cognitive processes, showing increased EEG activity following a blink.For example, Wascher et al., [8] discovered that blinking was associated with an increase in theta-band power in the EEG signal when there were higher demands for walking tasks.Another study by Liu et al., [38], using MEG, revealed that spontaneous eye blinks activate the precuneus region of the brain, which is responsible for environmental monitoring and awareness.This suggests that blinking plays a significant role in regulating attention and cognition.Taken together, these findings suggest that blinking is a neurophysiological process that actively contributes to regulating attention and engaging cognitive processes.
Our findings showed that blink-related neural activity following the blink maximum differed significantly between experimental conditions.The use of MVPA enabled us to identify the temporal dynamics and similarity of pro-and re-active level representations.The results revealed that levels of pro-active driving could be accurately distinguished above the chance level at temporally adjacent points following the blink maximum, likely due to the varying visual information presented at different levels of pro-active driving.In contrast, during the analysis of re-active driving, we found that only the comparison between low and high task loads could be significantly decoded above the chance level.This implies that decoding the neural mechanisms and visual processing associated with intermediate levels of re-active driving is challenging, possibly due to distinct learning experiences requiring comparable attentional resources and visual information.
However, the broader context of the EEG data, which extends beyond a specific temporal threshold (400 ms) from the peak of the blink, or the utilization of random EEG epochs, seems to carry a diminished cognitive signal.This raises questions about their potential relevance in capturing meaningful neural activity.The inability to decode random EEG epochs highlights the complexity of brain activity associated with different neurological events.These findings emphasize the precision in decoding blink-related EEG data while also revealing a noticeable reduction in decoding effectiveness when applying the same methodology to non-specific, random EEG epochs.
Returning to blink-related EEG data, within the realm of high-performance decoding, the human brain exhibits notable neural activity that appears to be specifically related to the processing of visual information.This is supported by the contribution of various electrodes, predominantly located in the parieto-occipital channels, to the decoding performance.These observations suggest that the brain is actively involved in visual information processing, in line with prior research that has linked the activation of the parieto-occipital sulcus region with voluntary blinks [38], [39], [40].In studying the spatiotemporal characteristics of ocular currents resulting from voluntary blinking in both light and dark conditions, these studies found that occipital activations only occurred in the light condition, indicating visual processing [40], [41].
Furthermore, the results also demonstrate significant decoding of pro-vs.re-active driving data as well as active vs. passive operators.Following the blink maximum, decoding peaks were observed at adjacent time points, implying that eye blinks consistently and transiently impact ongoing brain activity.These decoding peaks were found to occur approximately 200 ms after the blink maximum, indicating a predictable time lag.Furthermore, the study employed temporal generalization analysis, revealing a consistent processing pattern of specific neural processes or cognitive operations occurring at particular intervals following eye blinks.Therefore, regardless of the driving situation or operator type, eye blinks are likely to have a comparable influence on brain activity during this specific time interval.
Our study extends these findings by demonstrating that blinkrelated measures can effectively decode cognitive states during driving and steam engine operation.Surprisingly, the data collected from blink-related potentials provided sufficient information to distinguish between different conditions with accuracy above chance level, and accurately decode most conditions around the blink maximum.This finding emphasizes the valuable insights that blink-related potentials offer into cognitive processing, as evidenced by the high decoding performance observed when correlating with the variance difference of eye closure and opening between conditions.Previous studies have confirmed that variations in eye blink parameters can be induced by differences in cognitive or visual load [13].However, we assume that the visual load was comparable across all conditions, except for the low vs. middle and middle vs. high re-active driving conditions, where we believe the difference was not strong enough to induce significant blink modulation.
Integrating blink and related neural activity into cognitive and behavioral evaluations holds great potential.Our results also suggest that eye blinks can complement EEG measures in assessing cognitive states during driving, steam engine operation and other real-world scenarios.These findings align with previous research, which indicated the feasibility of utilizing eye blinks-based EEG as a non-invasive measure correlating with neural mechanisms underlying attentional demand [42].Hence, blink-related measures could serve as a substitute or a Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.
complementary tool alongside blink-related EEG data in various cognitive and behavioral assessments.
Despite the promising results of our study, further research is needed to explore the neuronal processes linked with eye blinking events and their patterns before and after blinking to understand their cognitive implications better.Also, our study only involved two experiments, which may limit the generalizability of our findings.Despite the controlled experimental settings, there may be variations in the real-world scenarios that were not captured in our study.Therefore, further studies are required to validate our findings in different contexts and populations.Additionally, the EEG-based decoding approach used in our study provides limited information about the neuroanatomical origin of rhythmic activity.EEG signals only provide information about the electrical activity of the brain's surface and do not give a clear picture of the underlying neural network activity.Future studies may consider combining EEG with fMRI or functional near-infrared spectroscopy to provide better integration of anatomical and temporal information.

V. CONCLUSION
This study offers valuable insights into the relationship between blinking, cognitive states, and neural activity during driving and steam engine operation.The results demonstrate significant differences in blink-related neural activity following a blink maximum between experimental conditions, suggesting that the underlying neural processes vary between conditions following blinks.Conversely, randomly selected EEG epochs, which are not locked to blinks, do not exhibit any significant differences from the chance level, underscoring the unique EEG patterns associated with eye blinks.The authors also emphasize the potential of blink-related measures in decoding cognitive states during driving and steam engine operation, and how these measures can complement EEG data in various cognitive and behavioral assessments.Integrating blink and related neural activity into cognitive and behavioral evaluations could enhance our understanding of attentional demand and the engagement of cognitive processes in real-world scenarios.These findings have significant implications for developing more effective and accurate assessments of cognitive and behavioral states during complex tasks such as driving and operating machinery.The results propose that blinking could serve as an essential pacing mechanism for processing visual information during driving and steam engine operation tasks, highlighting its critical role in regulating attentional processes in such tasks.

Fig. 1 .
Fig. 1.Average global field power (GFP) across all electrodes and participants for (a) re-active (low, middle, and high difficulty levels), (b) pro-active (low, middle, and high difficulty levels), and (c) steam engine operating conditions (active and passive).The colored lines indicate when the GFP significantly differed from pre-stimulus levels at p < 0.05 significance level for each difficulty level.

Fig. 2 .
Fig. 2. Decoding accuracy of EEG data between difficulty levels of re-active and pro-active conditions over time.Panel (a) shows the time courses of EEG data decoding accuracy, while panel (b) displays the temporal generalization maps for decoding across time and conditions using one-versus-one classification with border color lines representing between conditions decoding.The classifiers trained at each time point were tested on all other time points in the time series.The significant clusters obtained via the pixel-based permutation test are highlighted using black contour lines.Panel (a) also shows weight maps of EEG data channels contributing to high decoding performance at specific times, with the corresponding period of significant decoding at the group level marked by bottom red horizontal lines below the time courses (p < 0.05, FDR).

Fig. 3 .
Fig. 3. Decoding accuracy of blink data between difficulty levels of re-active and pro-active conditions over time.Panel (a) shows the time courses of blink data decoding accuracy with the corresponding period of significant decoding at the group level marked by bottom red horizontal lines below the time courses (p < 0.05, FDR).Panel (b) displays the temporal generalization maps for decoding across time and conditions using one-versus-one classification with border color lines representing between conditions decoding.The classifiers trained at each time point were tested on all other time points in the time series.The significant clusters obtained via the pixel-based permutation test are highlighted using black contour lines.

Fig. 4 .
Fig. 4. Decoding accuracy and temporal generalization matrices for the driving and steam engine data.Panels (a) and (b) display the decoding accuracy and temporal generalization matrices based on EEG and blink data, respectively, for pro-active vs. re-active driving and active vs passive operator.The temporal generalization maps are color-coded with border lines corresponding to the decoded conditions.At each time point, classifiers were trained and then tested on all other time points in the time series.The significant clusters obtained via the pixel-based permutation test are highlighted using black contour lines.

Fig. 5 .
Fig. 5. Relationship between blink behavior and high accuracy performance for driving and steam engine data.The normalized mean variance of eye closing and opening data across all trials for driving and steam engine data are displayed in panels (a) top and (b) top, respectively, providing a comprehensive overview of the variability between conditions.Scatterplots (a) bottom with Spearman's rank coefficients for driving and (b) bottom for steam engine data show the correlation between variance difference and high accuracy performance for eye opening and closing, and highlight the stronger correlations for eye closing (r = 0.38 for driving, r = 0.81 for steam engine) compared to blink opening (r = 0.17 for driving, r = 0.58 for steam engine).
Decoding Eye Blink and Related EEG Activity in Realistic Working Environments Emad Alyan , Stefan Arnau , Julian Elias Reiser , Stephan Getzmann , Melanie Karthaus , and Edmund Wascher