Exercise Fatigue Detection Algorithm Based on Video Image Information Extraction

Excessive psychological pressure, long working hours, and excessive labor intensity can make people exhausted and affect people’s cognition and motor function. Detecting the fatigue state of athletes can prevent excessive fatigue and sports injuries. This article chooses the adaptive median filter method to smooth the image and remove the noise, and uses the adaptive threshold light equalization method to adjust the image’s light equalization. According to the admission and rejection criteria of the Sequential Forward Floating Selection (SFFS) algorithm, different feature parameter combinations are used to build a fatigue motion detection model based on Support Vector Machine (SVM). Taking the classification performance of the built SVM detection model as the evaluation criterion, and using the sequence floating forward selection algorithm as the search strategy, the fatigue characteristic parameter optimization selection algorithm is established. The algorithm is used to reduce the dimensionality of the full set of fatigue feature parameters, and the optimal feature subset of fatigue motion is extracted. Based on the paired sample t-test and the analysis of variance method, it analyzes and quantifies the comprehensive influence of individual athlete differences and fatigue exercise on sports behavior and eye movement characteristics. An adaptive detection model is built based on personality parameters, and the design idea of the fatigue feature extraction network is analyzed. In order to make full use of the information of the feature vector output by the fully connected layer, the new network designs two fully connected layers to extract feature vectors. Two types are output by the Softmax loss function, which can directly determine whether the athlete is in a fatigue state. Based on the PERCLOS (Percentage of Eyelid Closure Over the Pupil over time) criterion, this article completes the construction of the fatigue motion sample set, and classifies the face images with more than 80% eyes closed as fatigue samples. This method can apply the PERCLOS criterion to the training of the convolutional neural network, so that it can recognize the fatigue state of the face based on the comprehensive facial features and improve the robustness of the algorithm.


I. INTRODUCTION
In the fields of aerospace, sports training, nuclear reactor operations, and deep-sea operations, the physical and mental state of personnel are high, and real-time physiological signal monitoring is performed to assess the fatigue state of personnel, prevent sports fatigue, fatigue operations, and prevent accidents, which can avoid casualties and property damage [1]- [3]. Therefore, it is necessary to monitor the sports fatigue of the above-mentioned personnel. The application The associate editor coordinating the review of this manuscript and approving it for publication was Zhihan Lv . of sports fatigue detection technology is widely used, and the research is of great significance [4]. Sports competitions require athletes to perform a lot of training, and increasing the amount of training will increase the probability of athletes contracting epidemic diseases, especially in the early stage of preparation for the athletes' psychological pressure and high training intensity [5], [6]. With the increase in exercise intensity, exercise fatigue will occur, which will affect the normal function of the human immune system and make the body weak and increase the prevalence [7], [8].
Mechanical exercises on the human body are prone to fatigue [9]. For people with high concentration, fatigue is more likely to occur during work. Researchers all over the world have spent a lot of time and energy on the study of sports fatigue and conducted in-depth research [10], [11]. However, due to the wide fields and complex correlations involved in sports fatigue, there has not been a unified and recognized result. Relevant scholars use the processing of Doppler radar and physiological signals to obtain fatigue parameters such as the athlete's mood change and blink frequency to determine whether the athlete is dozing off [12]- [14]. By making these devices into small devices and installing them above the athletes' heads, the athletes' normal sports activities will not be affected during the exercise [15]. Researchers have developed a device for detecting fatigue based on the physiological signals of athletes [16]- [18]. Its appearance is similar to today's popular smart bracelets and can be worn on the wrist [19]. The built-in sensor can measure the content of chemical substances such as lactic acid and ammonia in the athlete's sweat, and judge the athlete's mental state by comparing with a given value [20]- [22]. Once the athlete is detected to be fatigued, it will immediately sound a warning. As an objective indicator of sports fatigue detection, physiological signals have the characteristics of non-destructive, safe and convenient, and have become a hot field of real-time monitoring of sports fatigue [23], [24]. The bioelectric signal physiological indicators used for exercise fatigue estimation include signals such as myoelectricity, eye electricity, brain electricity, and ECG [25]- [27]. By monitoring the changes of bioelectrical signals during exercise, the characteristic indicators that are strongly related to exercise fatigue are selected for exercise fatigue assessment [28]. Relevant scholars collected the ECG and EMG signals of the experimenters in the exercise and fitness process, and analyzed the characteristics of the signals in the time domain and frequency domain respectively, and found the true characteristic indicators that reflect sports fatigue, and used Hidden Markov Model (HMM) to construct the HMM sports fatigue multi-classification model based on bioelectric signal [29]. Deep neural network can model multi-level fully connected network structure according to actual needs [30]- [32]. It has rich hierarchical structure and strong learning ability, which overcomes the limitations of pattern recognition. Related scholars used the Convolutin Neural Network (CNN) model for modeling in the emotional multi-classification problem, and obtained satisfactory results after testing on multiple data sets [33]. Researchers use long and short-term memory recurrent neural networks for sentiment analysis, making full use of the characteristics of recurrent neural networks that effectively use sentence structure information and long-and short-term memory neural networks [34]. The model was tested using the film review data set provided by Stanford University, and the accuracy rate was as high as 88.1%.
Based on the admission and rejection criteria of the Sequential Forward Floating Selection (SFFS) algorithm, this article uses different feature parameter combinations to build a fatigue motion detection model based on Support Vector Machine (SVM). Taking the classification performance of the built SVM detection model as the evaluation criterion, and the SFFS algorithm as the search strategy, an optimization selection algorithm for fatigue characteristic parameters is established. The algorithm is used to reduce the dimensionality of the full set of fatigue feature parameters, and the optimal feature subset of fatigue motion is extracted. Using the optimal subset as the input of SVM, a general fatigue motion detection model is built. Aiming at the difference of the optimal time window of different characteristic parameters, a data fusion method of sliding time window is proposed, which improves the real-time detection and realizes the online real-time detection of fatigue motion. Specifically, the technical contributions of this article can be summarized as follows: First: This article performs denoising filtering and light equalization on the collected video images. Good results have been obtained through MATLAB programming simulation. We choose adaptive median filter algorithm to filter and eliminate noise in the image. Compared with the traditional mean, median and Gaussian template filtering algorithms, this method can automatically filter according to different images. While removing noise, it also preserves image edge and detail information. The light equalization method based on adaptive threshold enables the dark images collected at night to be illuminated, and the bright images collected during the day are illuminated to balance, which solves the problem of uneven illumination.
Second: Based on the paired-sample t-test and analysis of variance, it quantifies the influence of individual athletes' individual difference factors on fatigue testing. Based on the athlete's own stability, the reference mean is extracted using normal exercise data, the personality parameters are calculated according to the characteristic parameters, and an adaptive detection model is built using the personality parameters. In the early stage of exercise, the universal detection model is used to detect and initialize the adaptive detection model, and then the adaptive detection model is used to detect the fatigue state of athletes.
Third: Compared with related research based on traditional machine learning methods, the algorithm in this article has achieved a certain lead in outdoor environments. Compared with the research using the same convolutional neural network technology, this article breaks through the limitation of the fatigue feature extraction based on pupil opening, adopts the method of Finetune, reuses the facial feature extraction network, and makes the fatigue feature detection module of the algorithm. The full face information can be extracted as the input source, which improves the accuracy and robustness of the algorithm. From the experimental results, the detection effect of the algorithm in this article has a certain decline in the face of strong outdoor interference. The main reason is that the athlete's head posture changes exceed the corrective ability of the preprocessing algorithm. When processing some image frames with strong posture changes, the Histogram of The Oriented Gradient (HOG) algorithm and the Constrained Local Neural Fields (CLNF) algorithm show the phenomenon of target loss and inaccurate positioning. It can VOLUME 8, 2020 be seen that the algorithm design of the preprocessing module still has room for improvement.
The rest of this article is organized as follows. Section 2 analyzes the video image preprocessing and edge detection methods. Section 3 constructs a fatigue motion detection model. Section 4 analyzes the experimental results. Section 5 summarizes the full text.

II. VIDEO IMAGE PREPROCESSING AND EDGE DETECTION
A. IMAGE FILTERING AND DENOISING Image filtering is a commonly used method in digital image processing. It usually refers to the method of removing some useless information according to the needs of image processing and recognition, and retaining the useful information in the image. Its main purpose is to make the processed images more suitable for human observation and machine recognition systems for certain specific applications. When the image is disturbed by signals such as noise, it will make the image blurry. In severe cases, it will destroy the image information and affect further face detection and fatigue state recognition. Therefore, it is necessary to remove noise interference and improve image quality through spatial or frequency domain image filtering. The schematic diagram of the non-subsampled Contourlet transformation is shown in Figure 1. A digital image can be regarded as a two-dimensional space function f(x, y), (x, y) represents the position of the pixel in the image space, that is, the spatial domain. Spatial filtering is mainly based on the calculation of the neighborhood of each pixel and the response output is obtained. The process of spatial filtering is to move the template point by point in the two-dimensional image to make the center of the template coincide with the pixel points. When the template is located at the edge of the image, it will cause some elements of the template to exceed the image boundary. This type of problem can be solved by supplementing the image by shrinking the processing range, filling the image with constants, and duplicating pixels. Image filtering techniques are roughly divided into two categories: one is spatial domain filtering, and the other is frequency domain filtering.
Since the gray value of the noise pixel in the digital image is very different from its neighboring pixels, the traditional median, mean or Gaussian template filtering method can be used to filter and denoise the collected video image.
Mean filtering is neighborhood average, which is a linear smoothing filter and uses the average value in a pixel neighborhood as the filtering result output. We select a certain size neighborhood for each pixel in the image, and replace the central pixel value with the average gray value of the pixels in the area. We use the template to traverse all the pixels in the image to obtain the filtered image. The specific algorithm is: Suppose an image f(x,y) is an M×N array, the smoothed image is I(x,y), and the gray scale of each pixel consists of a fixed-size area containing (x,y). To determine the gray average of all pixels within, its definition can be expressed as: The mean filtering method has a better effect on removing Gaussian noise, but it has certain limitations at the cost of blurring edges and details. Median filtering is a non-linear spatial smoothing method widely used to remove impulse noise (such as salt and pepper noise). Its essence is a statistical sorting filter. This method arranges all pixel values in a pixel area in ascending order, and takes the gray value at the middle position as the filter output response. This method can make the center pixel that is very different from the surrounding pixels closer to the surrounding pixels, so as to achieve the purpose of denoising and smoothing.
In order to better preserve the details and edges in the image after filtering, the Gaussian filtering method adjusts the weight of the filter template to make the weight of the center point larger, and the weight far away from the center of the template becomes smaller. You ensure that each point pixel is closer to the point closer to it after filtering. Based on this consideration, the Gaussian filter template is obtained. The Gaussian template discretizes the continuous two-dimensional Gaussian function in the spatial domain. Therefore, the Gaussian filter template of any size can be represented by a (2k+1) * (2k+1) odd-order matrix M(i, j). The element value at the position (i, j) can be expressed as: The value of θ is very important. It determines the weight of pixels off the center. If the value is too small, the filtering operation will degenerate into a point operation. If the value is too large, the filter will degenerate into a mean template.
Although the above three methods can achieve a certain denoising effect, the size and parameters of the filter template need to be set in advance. As the actual application will be interfered by different factors as the environment changes, it is necessary to find an adaptive filtering method to improve the system's ability to automatically filter and process video images.
Considering that although the Gaussian template reduces blur and distortion to a certain extent, the size of the filter template is fixed each time. Therefore, this article chooses adaptive smoothing filtering to selectively deal with image noise, that is, no smoothing filtering is performed in the area without noise, so that the influence of noise and blur can be minimized.

B. IMAGE LIGHTING EQUALIZATION BASED ON ADAPTIVE THRESHOLD
Because the brightness of the video images of the athletes collected under different lighting environments is different, if the video images are too bright or dark, it will affect the effect of face detection, and it is not conducive to the recognition of the state of eyes and mouth. This article uses the light equalization method based on adaptive threshold to decompose the image into YCbCr space, analyze the characteristics of the image in the subspace, find the reference white point and adjust it. We convert the image from RGB space to YCbCr space, the conversion formula is as follows: Secondly, we calculate the average values of Cb and Cr, Mb and Mr, and calculate its mean square error Db, Dr by the following formula.
For the sub-image division area, if the mean square error of a certain area is too small, it means that there is no obvious color change in the area, and this part needs to be discarded. Through the judgment method of the following formula, all pixels close to the white area are obtained.
According to the characteristics of the brightness value, the pixels close to 8% of the white area are selected as candidate reference white points. You obtain the channel gain value from the reference white point. In order to maintain the brightness of the entire image unchanged, the channel gain can be obtained from the maximum brightness. The calculation method is as follows: Among them, Y max is the maximum brightness of all pixels in an image, and R ave , G ave , and B ave are the average values of the reference white point in the R, G, and B channels. Finally, you adjust the picture through the following formula to achieve the white balance effect of the processed image.
Image segmentation is generally used to extract regions or structures of interest in an image. The segmentation method is generally based on the discontinuity and similarity of the brightness value. Discontinuity refers to the use of sudden changes or step changes of pixels in an image to classify and extract pixels belonging to different gray value ranges. The edges of objects in the image often exhibit this characteristic. The second characteristic, similarity, usually divides the image into similar regions, and the gray value and texture structure inside the object often show this characteristic. They use methods such as threshold processing, region separation and aggregation to mark or extract these similar regions from the original image. In general, it is necessary to formulate corresponding criteria based on the pixel characteristics of the target area in the image to extract the target area.
In the field of digital image processing, the gray value range of pixels in a single-channel image is generally an 8-bit integer value of 0∼255, where 0 and 255 correspond to all black and all white in black and white images, respectively. The gray histogram is used to count the number of pixels of each level of brightness in the image and plot them in a histogram. The target area is significantly different from the background gray value. The gray value of the foreground and background can be clearly observed by using the gray histogram, so as to use threshold segmentation method to segment the image.
Single threshold segmentation, also called threshold processing, is an intuitive and easy-to-implement image segmentation method. It plays a central role in related applications of image segmentation. For pictures with obvious gray difference between foreground and background, a threshold value is usually selected according to the gray histogram to separate the foreground and background regions. In the following formula, (x, y) is the coordinates of the pixel point in the image, and its gray value is f(x, y), f'(x, y) is the gray value corresponding to the (x, y) point after threshold segmentation, M is the gray value of the extracted target area.
Adaptive threshold segmentation is a threshold segmentation method developed on the basis of single threshold segmentation. It calculates and applies multiple thresholds in the same image to adapt to the target area of the same object in the image due to the shooting angle and other reasons. Simple VOLUME 8, 2020 adaptive threshold segmentation calculates the average gray value of the target pixel area, and adds it to a constant set by the user to obtain the threshold value of the pixel point, and then uses the threshold value for segmentation processing. The difference between the weighted adaptive threshold and this method is that the calculation is not the simple average of the pixels in the field, but the weighted average of the pixels in the field according to the distance between themselves and the center point. Generally, a Gaussian kernel function is used to calculate a weighted average in a central pixel area to perform threshold segmentation. The adaptive threshold technology has ideal results when thresholding images with strong exposure or reflection gradients.

2) CANNY EDGE DETECTION
Commonly used edge detection algorithms include searchbased algorithms, zero-crossing-based algorithms, Canny edge detection algorithms, and statistical discrimination methods. Among them, the search-based method first calculates the first derivative of the image, and then finds the positions where the maximum and minimum values of the first derivative appear to determine the image boundary, which is a simple and intuitive edge detection method. The method based on zero crossing is based on the calculation of the first derivative and the zero-crossing point of the second derivative. Its essence is basically the same as that of the search-based method. Both are gradient-based edge detection operators. The schematic diagram of video image edge detection based on local hybrid filtering is shown in Figure 2.
The edge detection algorithm increases the uncertainty of the edge when filtering the image, and the edge information becomes not very obvious, thereby affecting the result of edge extraction. The Canny edge detection operator tries to balance the two aspects of anti-noise interference and precise positioning of the edge of the object. The Canny algorithm first obtains first-order derivatives along the horizontal and vertical directions in the original image, and selects the points where these derivatives reach the local maximum as candidate points of the edge, and then assembles these candidate points into a contour, that is, the edge of the object.
The Canny algorithm first obtains first-order derivatives along the horizontal and vertical directions in the original image, and selects the points where these derivatives reach the local maximum as candidate points of the edge, and then assembles these candidate points into a contour, that is, the edge of the object. The algorithm is specifically divided into the following four steps: In the first step, the image is smoothed and filtered by the Gaussian smoothing filter, that is, the image is convolved using the Gaussian convolution kernel. The reason for using the Gaussian smoothing filter here is that the Gaussian kernel is a kernel with weights, which can retain the original edge information in the image relatively well.
The second step is to calculate the size and direction of the gradient in the image. Here, the gradient is calculated using a first-order difference convolution template.
In the third step, the maximum point of the gradient in the region can be obtained by comparing the magnitude and direction of the gradient, and the obtained gradient of the non-maximum point is set to zero, thereby highlighting the edge of the maximum value.
In the fourth step, the non-zero values obtained in the previous step are connected into a complete contour. In this step, a double threshold algorithm is used. Usually the algorithm will automatically select two thresholds T1 and T2 (T2> T1) to binarize the edge contour obtained in the previous step to obtain two edge images N1 and N2. Since the threshold used by N2 is higher than that of N1, the edges in N2 can be considered as true edge residues. But the edge pixels in N2 are usually not continuous, so the algorithm needs to find the endpoints of the contour in N2, and then use the contour information in N1 to stitch the contour. The specific method is to find the 8 field position of the pixel corresponding to the left position in N1 on the endpoint, until N2 is spliced into a complete contour.

III. FATIGUE MOTION DETECTION MODEL A. SELECTION AND OPTIMIZATION OF FATIGUE CHARACTERISTIC PARAMETERS 1) FEATURE SELECTION ALGORITHM
Feature selection is also called attribute selection. With the development of data mining technology, feature selection technology has been paid more and more attention in many fields and has been widely used. Feature selection refers to removing irrelevant features and removing redundant features from the original feature set, so that the selected feature subset is the optimal feature subset, so as to improve the accuracy of the fatigue motion detection model, or to ensure the complexity of the model structure under the condition of a certain prediction accuracy. The framework of the classifier training system based on video image information extraction is shown in Figure 3.
The Filter method mainly performs feature screening by measuring the degree of correlation between features and class tags. This method is simpler than the encapsulation method, but it cannot deal with the feature redundancy problem. According to whether they are scoring a single feature or a subset of features, they can be divided into one-element Filter method and multiple-element Filter method.
The advantage of Filter technology is that it can efficiently solve the data screening problem, the calculation is simple and fast, and the evaluation criteria for feature selection are independent of the classifier. Therefore, the feature selection operation of the Filter algorithm only needs to be executed once, and then the dimensionality reduction data can be used to construct a classifier and evaluate the classification accuracy. A common disadvantage of Filter methods is that they ignore the interaction with the classifier, which means that each feature is considered independently and therefore ignores the correlation of features, which will result in worse classification performance compared to other feature selection techniques. In order to solve the problem of ignoring feature correlation, many multivariate filter technologies have been introduced, aiming to incorporate the consideration of feature dependence to a certain extent.
The Wrapper method is a very straightforward way to filter a subset of features. Unlike the Filter technology, which deals with finding a feature subset independent of the model selection step, the Wrapper method embeds the hypothesis space of the model into the search of the feature subset, combines it with a specific classifier, and cross-validates the feature subset. The classification accuracy is evaluated on the set, and a set of feature subsets with the highest classification accuracy is selected as the optimal feature subset. At the beginning of the algorithm, a process of searching in the space of possible feature subsets is defined, and then many feature subsets are generated and evaluated. This evaluation of a specific feature subset comes from a specific classification model, which is applied to each step of training and verification. However, when the feature subset increases exponentially with the number of features, the cost of exhaustive search becomes extremely high. Therefore, heuristic search methods are usually used to guide the search direction of the optimal subset.

2) SFFS ALGORITHM
Sequential Forward Floating Selection (SFFS) is one of the classic methods in the Wrapper method and belongs to the local optimal algorithm. SFFS methods mainly include Sequential Forward Selection (SFS) and Sequential Backward Selection (SBS) steps to achieve its search goals. Among them, the SFS method first makes the target data subset empty, and then adds features to the data set in order to form a target subset set, and evaluates the target subset set according to the discriminant criteria, and renews it after each evaluation; the SBS algorithm deletes the previously formed data set one feature each time to form a new feature set, evaluates it according to the evaluation criteria, and then selects the search direction based on the evaluation results. The SFFS algorithm integrates the SFS algorithm and the SBS algorithm. Its advantage is that every time a data subset is generated, certain relatively unimportant features are dynamically deleted, which can reduce the search order to a certain extent.
The fatigue feature optimization selection algorithm established in this article can be transformed into an optimization problem, as follows: Among them, Y is the full set of fatigue characteristic parameters, X is a non-empty subset of Y, J(X) is the criterion function, that is, the detection accuracy of the fatigue detection model, N X is the number of test samples, n X is the correct fatigue motion detection model in the test sample.
The basic idea of fatigue feature subset selection based on the SFFS algorithm uses the SFFS algorithm to search for a non-empty subset X from the full set of fatigue feature parameters Y, and then uses X as input to build a fatigue motion detection model based on support vector machines and uses test samples to find the value of the criterion function J(X). Finally, you select the subset X corresponding to the maximum value of J(X) as the optimal feature subset.
You select the parameter x k+1 from the set Y −X k to maximize the criterion function value J(x k+1 ) after selecting x k+1 : You remove the parameter x i from the selected parameter set X k to maximize the criterion function value after removing x i : The SFFS algorithm starts from the empty set, and selects subset Z from the unselected parameters in each round, so that the criterion function after adding the subset Z is better; then selects the subset S from the selected indicators to eliminate the subset S.

3) CONSTRUCTION OF FATIGUE MOTION DETECTION MODEL
The blinking frequency of athletes per minute BF is generally between 7-25, and the size of P80 is generally between 0-0.4, and the two are quite different in order of magnitude. When solving the optimal classification surface of the fatigue motion detection model, the feature parameters with a large order of magnitude will dominate the small feature parameters, causing the small feature parameters to be submerged, thereby weakening its contribution to the classification. The order of magnitude is unified to eliminate the dimensional difference, that is, normalization. The normalization of feature parameters can improve the detection accuracy of the fatigue motion detection model based on SVM, and can reduce the search range of variable optimization and shorten the training period. This article uses the following formula to normalize the components of the fatigue characteristic parameter subset X: Among them, x is the original feature parameter component; x max and x min are the maximum and minimum values of the original feature parameters of the training sample; N is the total number of the training sample set.
Because the migration law of athletes' fatigue state is complex and not a simple linear problem, it is necessary to introduce a kernel function to map it from the original input space to the high-dimensional feature space.
The radial basis kernel function has certain advantages in dealing with the fatigue state detection of athletes. In this article, the RBF kernel function is selected as the kernel function of the fatigue feature subset from the original space to the high-dimensional linear space. There are two undetermined variables in the model building process: the penalty coefficient C and the nuclear variable γ . Among them, the penalty coefficient C controls the recognition accuracy and generalization ability of the athlete's fatigue detection model, and the nuclear variable γ determines the linear separability of the nonlinear problem in the original space after being transformed into the high-dimensional space. The purpose of variable optimization is to determine the appropriate C and γ to ensure the recognition accuracy and generalization ability of the fatigue motion detection model. In this article, the recognition accuracy of the model on the test set is used as the objective function of the optimization of the two undetermined variables, and the grid-search method is used to search for the optimal combination of variables.
The cross-validation method is used to obtain the objective function value of model variable optimization and index evaluation criteria. The cross-validation method divides the complete sample set into k randomly. The ratio of the sum of the number of samples to the total number of samples is used as the classification accuracy of the model. In the specific implementation, this article takes the value of k as 5, and uses the 5-fold cross-validation method to obtain the recognition accuracy of the fatigue detection model, which is used as the criterion function value for variable optimization and index screening.

B. ADAPTIVE DETECTION MODEL
The individual differences of athletes limit the accuracy of fatigue detection. However, for the same individual, the fatigue characteristics show considerable self-stability at different times and locations. Therefore, it is possible to conduct online learning of the individual characteristics of the athletes when they are awake, and then detect the state transition process of the athletes according to the changes of these individual characteristics during the exercise. Based on this, this article proposes an adaptive detection method for athlete fatigue, which eliminates the influence of individual differences and improves the recognition accuracy and generalization ability of the detection model.
Relative to itself, the sports style and sports proficiency of the same athlete has its own stability. Therefore, the data in the sample database of each athlete during normal exercise is used as the reference data to calculate the optimal feature subset under the normal exercise state. The reference mean R ij of the 10 parameters P ij is: Among them, P ij is the jth characteristic parameter of the ith athlete, and R ij is the reference mean value of the jth characteristic parameter of the ith athlete under normal sports conditions.
The calculated value of each characteristic parameter and the average value R ij of the characteristic parameter during the normal exercise of the athlete are calculated to obtain the personality parameter PP ij of the characteristic parameter of the athlete. The calculation formula is: You use all the personality parameters PP of all athletes to construct the training sample library of the fatigue exercise adaptive detection model. And according to the fatigue exercise construction process, the SVM-based adaptive fatigue exercise detection model is established.
In the first 20 minutes of the start of the exercise task, the general fatigue exercise detection model is used to detect the fatigue state of the athletes. If the athlete is in a normal motion state, you extract the athlete's motion behavior and eye movement behavior data during this period, and calculate the athlete's reference mean R ij . Then, the slip time window method is used to calculate and integrate the athlete's personality parameter PP as the input of the adaptive detection model to realize the online real-time detection of the athlete's motion state. The detection process is shown in Figure 4.

C. FATIGUE MOTION FEATURE EXTRACTION NETWORK DESIGN
In order to ensure the smooth progress of the parameter migration of Finetune technology, the fatigue motion feature extraction network must keep its underlying structure consistent with the prototype network.
The Openface network outputs a 128-dimensional feature vector. This vector is an abstraction of high-level features of a human face. It contains facial information needed for athlete fatigue discrimination, but it cannot directly classify the athlete's fatigue state, so the design goal is to integrate the 128-dimensional vector group into a two-dimensional vector, and directly use the athlete's face picture for fatigue determination.
In the design of the rest of the athlete fatigue detection network, considering that the output of the eighth layer of the network is a 128-dimensional feature vector, the extracted facial features are sufficiently simplified, and the ninth layer of the network should not be designed as a pooling layer. The pooling layer has the characteristics of fuzzy input. After the convolutional layer, it has the effect of reducing network parameters. However, at this time, how to ensure that the information of the 128-dimensional vector does not attenuate is the primary consideration. The attenuation of key information cannot be ignored.
The ninth layer of the network should not be designed as a convolutional layer, because the convolutional layer has the ability to extract the inherent characteristics of structured data, and the 128-dimensional vector is too small for the convolution input, and its internal elements often exhibit sparse characteristics. In this case, adding a convolutional layer will usually lead to non-convergence during network retraining.
In summary, the new fatigue detection network should add two fully connected layers on the basis of Openface. The ninth layer is designed with 32 nodes, and the 128-dimensional vector is compressed to 32-dimensional. It does not use the pooling layer and Dropout technology. You ensure that information is not lost. The tenth layer is designed with two nodes, using the Softmax loss function, so that the new network can directly output the results of fatigue detection, that is, whether the athlete is in a fatigue state.

A. EXPERIMENTAL RESULTS OF VIDEO IMAGE PREPROCESSING 1) EXPERIMENTAL RESULTS OF IMAGE FILTERING AND DENOISING
According to the algorithm principle of adaptive median filtering, this article uses MATLAB software to simulate the filtering denoising results shown in Figure 5. For each filtering and denoising method, the peak signal-to-noise ratio PSNR, mean square error MSE and running time T before and after image processing are calculated respectively, and objective evaluation indicators are used to further verify the effect of filtering and denoising, as shown in Figure 6.
From the comparison results of objective comprehensive evaluation indicators, it can be seen that the adaptive median filter algorithm can automatically adjust the filter parameters according to different noises in the image, which can not only filter the noise interference better, but also retain the edge and detail information in the image.

2) EXPERIMENTAL RESULTS OF ADAPTIVE LIGHT EQUALIZATION
The experimental results obtained according to the image light equalization method of adaptively adjusting the threshold are shown in Figure 7. It can be seen that whether it is a dark image taken at night or a bright image taken during the day, the adaptive threshold light equalization algorithm can automatically equalize the color and brightness in the image, and solve the problem of uneven lighting. VOLUME 8, 2020

B. TRAINING OF ATHLETE FATIGUE DETECTION NETWORK
PERCLOS (Percentage of Eyelid Closure Over the Pupil over time) fatigue detection standard is a common set of judgment standards in fatigue exercise research. In a specified time range, the percentage of the time that the human eye is closed to the total time is highly correlated with the person's fatigue. Counting the ratio of the time the human eye is closed to the total time in a certain period. Whether it is fatigue, because the image acquisition frame rate is constant, the duration ratio can be simply approximated by the image frame ratio. Figure 8 shows the change curve of eye closure in a 300-second period.
The database used in this article is collected by the experimental team itself, which contains 8000 face images collected from 160 volunteers, and the ratio of male to female is about 1:1. The face orientation, eye opening, mouth opening, and whether to wear eyes and other expression states in each image are slightly different. The images are collected under different lighting conditions.
After performing face pose correction on the images in the database, the PERCLOS P80 standard is used as the classification standard for positive and negative samples to divide the sample set into two categories. The positive sample set is non-fatigue face images, the number is 6000, and the negative sample set is 2000, and the ratio of positive and negative samples is about 1:1. The remaining pictures are used as the test set. Since the images used in the original training process of the Openface network and the images used in this database may be different in terms of light, skin color, etc., in order to avoid the effect of the migration parameters during network training, it is necessary to complete the construction of the sample set, and replace the original mean file with the new mean file. After the mean file replacement is completed, the network is modified according to the design content of the athlete fatigue detection algorithm.
The training process is shown in Figure 9. After 100 batches of training iterations, the accuracy of the network stabilizes at about 94%, indicating that the convolutional neural network training is completed.

C. FATIGUE MOTION DETECTION ALGORITHM EXPERIMENT
This section introduces the experimental results of the algorithm and compares it with similar studies in recent years. It discusses the advantages and disadvantages of convolutional neural networks in the field of athlete fatigue detection, and studies the causes of the algorithm's failure in the outdoor complex environment. In the athlete fatigue detection algorithm, the fatigue recognition accuracy rate of a single frame image determines the overall fatigue recognition accuracy rate of the algorithm. Therefore, the process of calculating the P value is omitted in this experiment, and the single frame image is directly used for the experiment to find the correct rate of fatigue recognition of frame images.
The experimental design is as follows: we record the motion experiment video separately in indoor and outdoor environments and decompose it into separate image frames, perform algorithm detection, compare the detection results with the manual annotation results, and calculate the sensitivity TP/ (TP +FP) and specificity degree TN/ (TN +FN).
These two values respectively represent the accuracy of the network to detect fatigue images and the accuracy of non-fatigue images. The final result of the indoor test is shown in Figure 10. A total of 6000 frames of awake face images and 2000 frames of tired face images were collected to test the algorithm. Compared with the results of manual annotation, the average discrimination rate of the algorithm is about 93%. This result is similar to the accuracy of the Openface network for facial feature extraction, indicating that the facial features extracted by the Openface network are effectively used in athlete fatigue detection after multiplexing.
In order to illustrate the advantages of this algorithm over traditional fatigue detection algorithms, the experimental results are compared with the recognition rates of other methods, and the results are shown in Figure 11. The network designed in this article is based on the full face extraction of fatigue features, using the overall change information of the facial features of the athlete when fatigued, and is not limited to a single pupil opening feature, so the comprehensive accuracy rate is slightly higher. In summary, the fatigue feature detection method based on convolutional neural network has certain advantages over the traditional feature recognition method, and the feature extraction method based on the whole face also has better performance than the feature extraction based on the pupil.
The outdoor test results of the algorithm are shown in Figure 12. A total of 6500 frames of awake images and 1500 frames of fatigue images were collected for testing. By comparing Figure 10 and Figure 12, it can be seen that the accuracy of outdoor test results has a certain degree of decline compared with indoor test results.
In order to illustrate the advantages of the algorithm in this article over the traditional fatigue detection algorithm, the experimental results of this article are compared with the accuracy of the gray projection method + Markov chain. The result is shown in Figure 13. The pupil is detected by the gray projection method + Markov chain, the pupil opening is used for fatigue detection, and the mixed test is performed indoors  and outdoors. The accuracy rate is generally lower than the algorithm described in this article.
The outdoor test results show that the interference generated by the outdoor environment still brings greater difficulties to the algorithm. There are many reasons for the decline of these two accuracy indicators. Tracking the algorithm processing flow reveals that excessive head shaking of athletes is the main reason for the decrease in accuracy of the algorithm, as shown in Figure 14.

D. FATIGUE EXERCISE ADAPTIVE DETECTION EXPERIMENT
Due to different sports styles and sports proficiency of different athletes, the sports behavior of the athletes is affected, and there are certain differences in the fluctuation rules of the operating behavior variables and vehicle state variables of different athletes. At the same time, the eye movement characteristics (blink frequency, gaze pattern, etc.) of different athletes under the same motion state also have certain differences. Therefore, the sports behavior characteristics and eye characteristics of athletes are not only affected by the state  of motion, but also affected by the individual differences of the athletes. The individual differences of the athletes will limit the detection results of the general fatigue detection model.
In order to test the influence of individual differences on the fatigue motion detection model, the general fatigue motion detection model built was used to detect the motion status of 20 athletes. The detection results are shown in Figure 15.
In order to deeply analyze the influence of individual differences on the fatigue detection of athletes, we take blinking frequency BF as an example to further compare the influence of individual differences and fatigue levels on characteristic parameters of athletes.
There are significant differences in the blinking frequency BF of the same athlete under different sports conditions. However, due to individual differences, the trend and range of change between different athletes are different. Figure 16 shows the distribution characteristics of blink frequency BF   of different athletes in the same exercise state. It can be seen intuitively that there are obvious differences in the blink frequency BF of different athletes in the same sport state.
There are significant differences in the blink frequency BF of the same athlete under different sports conditions, and there are also significant differences in the BF of different athletes in the same sports state, and the range and trend of changes in the BF of different athletes from normal exercise to fatigue exercise are also certain differences. Therefore, the same detection model is not suitable for all athletes. When constructing a fatigue motion detection model, individual athlete differences are also one of the factors that must be considered. In order to test the effect of the adaptive detection model on the fatigue motion detection of different athletes, the adaptive detection model was trained with the data samples of 19 athletes, and the data of the remaining 1 athlete was used to detect the continuity of normal and fatigue states in real time. The detection results are shown in Figure 17.

V. CONCLUSION
The facial area of the athlete will be affected by different lighting during the movement, and the video image will produce noise and blur during the acquisition and transmission process. Therefore, this article performs filtering and denoising and light equalization processing on the image in advance to ensure the accurate detection of the face area. Taking the classification performance of the built SVM detection model as the evaluation criterion, and the sequence floating forward selection algorithm as the search strategy, the fatigue feature parameter optimization selection algorithm is established, and the optimal feature subset of fatigue motion is extracted. Taking the optimal subset as input, a general fatigue motion detection model is built. Aiming at the difference of the optimal time window of different characteristic parameters, a data fusion method of sliding time window is proposed to realize the real-time detection of fatigue motion. Based on VOLUME 8, 2020 the paired sample t-test and analysis of variance, the influence of individual athletes' individual difference factors on fatigue detection is quantified. Based on the athlete's own stability, the reference mean is extracted using normal exercise data, the personality parameters are calculated according to the characteristic parameters, and an adaptive detection model is built using the personality parameters. In this article, a 10-layer convolutional neural network is designed to realize the function of extracting and identifying athlete fatigue features. Reasonably we use Finetune technology to reuse the parameters of the Openface network, migrate them to the new network to complete the training, overcome the difficulty of a small fatigue motion sample library, and improve the accuracy and robustness of the algorithm. The traditional fatigue detection algorithm based on pupil opening is less robust because it uses a single feature for fatigue discrimination. Based on the P80 standard in the PERCLOS criterion, this article completes the construction of the fatigue motion sample set, and classifies the face images with more than 80% eyes closed as fatigue samples. This method can apply the PERCLOS criterion to the training of the convolutional neural network, so that it can recognize the fatigue state of the face on the basis of the comprehensive facial features and improve the robustness of the algorithm.
Although the exercise fatigue detection algorithm studied in this article can identify the fatigue state to a certain extent, the research stage is still in the simulation of algorithm research and optimization. At present, the related algorithm implementation process still needs to be further implemented through the platform in an embedded manner. In the process of real-time detection, it is still necessary to consider problems such as complex environment, uneven illumination, and partial occlusion of the face, so as to further optimize and improve the algorithm in the article.