Electric Wheelchair Control Using Wrist Rotation Based on Analysis of Muscle Fatigue

The wheelchair is one of the most common assistive technologies for people with motor impairment, due to its environmentally friendly features in terms of mobility and comfort. However, the operational method for the conventional wheelchair is still inconvenience for people with finger problems. Therefore, in the project, a novel control method to operate an electric wheelchair, using hand gestures (wrist rotation) is developed. Sixty-five (65) participants were involved in this study. Five hand gestures were considered and studied for forward, backward, right, left, and stop maneuvers while considering the human ergonomics factor. In this study, the stop maneuver was determined based on the most comfortable hand position to mitigate a fatigue experience, with two gesture classifying methods further investigated. The first method based on threshold has a promising accuracy of 96% and 91% precision. This method, however, requires a calibration every time a new user is introduced. The second method, a Naïve Bayes approach, was observed to solve the problem, as it has about 99% of both accuracy and precision. The evaluation of this method was then conducted with six participants that operated the wheelchair to follow the trajectories from start to end. The results showed that the participants comfortably controlled the developed wheelchair system to the goal without any collision. Results from experiments indicate that the proposed approach has high accuracy and the potential to solve the problem related to finger dependencies and hand fatigue.

tive tools for people with special needs [2]. But in some 23 The associate editor coordinating the review of this manuscript and approving it for publication was Bernardo Tellini . cases, a conventional wheelchair that is operated by fingers 24 causes dilemma for users with finger problems. Quadriplegic 25 patients who could not control their legs and arms properly 26 are an example of such condition. They face difficulties in 27 their daily activities, such as eating and toilet usage. It is 28 also difficult for them to control a wheelchair using their 29 fingers [3], [4]. 30 Based on these conditions, an alternative method to con-31 trol wheelchairs is required to improve access for disabled 32 patients in doing their activities. Nowadays, the conventional 33 Another issue arising from controlling with the hand is 90 muscle fatigue, which causes lower precision in control abil-91 ity [33]. This leads to unstable hand positions [34], therefore, 92 increasing the noise. One of the experiments conducted to 93 determine hand muscle fatigue is by keeping the hand in a 94 static position [35]. This condition should be considered in 95 controlling the machine by hand [36]. So in this study, a new 96 mechanism using a hand wrist is developed to control an elec-97 tric wheelchair. Several hand positions are further studied to 98 determine the usual location for the stable operational hand. 99 In addition, a leap motion sensor is adopted and used to obtain 100 hand features, such as pitch, yaw, and roll parameters.

102
A wheelchair is an object controlled by users in this study. 103 Some researchers developed alternative methods to control 104 wheelchair, such as using a touch screen with a graphical 105 user interface [7]. A study on controlling a wheelchair using 106 leap motion is also reported [34]. The method, however, still 107 requires fingers to control the wheelchair properly, which can 108 be problematic for patients with finger problems. Therefore, 109 in this project, we develop a control system for a wheelchair 110 based on wrist orientations/movements. The system is devel-111 oped by considering muscle fatigue due to exhaustion. It is 112 set for the hand position to be in a natural position, with 113 minimum effort to withstand the gravity force for the no 114 movement/maneuver settings. A Naïve Bayes algorithm is 115 developed to recognize five hand gestures for five maneuvers: 116 forward, backward, right, left and stop.

118
There are four main stages in this study as shown in Fig. 1. 119 An electric wheelchair was designed with additional fea-120 tures such as hand sensors and a user interface monitor 121 to control the machine. Also, the hand gesture recognition 122 method was developed after the selection of the hand position, 123 which was studied based on its natural and relaxed level. 124 Subsequently, the hand gesture recognition methods and 125 performances were investigated and analyzed, respectively. A. ELECTRIC WHEELCHAIR SYSTEM

130
A system used to operate and control a motorized wheelchair 131 through hand gestures (i.e., wrist rotation/movement) was 132 designed in this study. The positions of the hand gestures 133 are captured by a sensor system as the input. The posi-134 tion is represented by pitch, yaw and roll. A graphical user 135    can cause a low accuracy performance [8]. 175 The control of the proposed wheelchair was based on the 176 position determined from the wrist orientation, which was 177 obtained using the leap motion coordinate. Furthermore, the 178 wrist orientation was defined by three variables, pitch, yaw, 179 and roll, along the x, y, and z-axes, respectively, as illustrated 180 in Fig. 6.

181
The hand position was initially investigated for the stop 182 button on the GUI application and considered as the reference 183 point for other gestures in controlling the wheelchair. It was 184 also found to be the resting point for users. Consequently, 185 the hand position that is easily formed and maintained for a 186 certain period should be used. Such a position should be able 187 to mitigate errors due to muscle fatigue.

188
A series of investigations to determine the position for the 189 stop point were further designed, with three candidates for 190 comfortability shown in Figure 7(a) [19], (b) [39] and (c). 191 The coordinates of these hand positions were captured by the 192 leap motion and set as the reference. The pitch, yaw, and roll 193 variables were also relative to these coordinates.

194
The number of participants (m) was calculated using 195 Eq. 1 following a reference by I. Guyon et al. (1) [40]. 196 The number was calculated with a 90% level of confidence, 197 z α = 1.28 and 20% β-error. Data from forty (40) participants 198 were recorded and used to train the developed algorithms. 199 Another twenty-five (25) participants (not included in the 200 training phase) were used in testing the algorithm.
They were all identified without a problem with their 203 hand and leg conditions. The abilities of these participants 204 to maintain the hand position based on the three wrist orien-205 tations were recorded. The data collected was also repeated 206 thrice for each position. Moreover, a subject is considered to 207 be tired when consciously becoming uncomfortable. In this 208 case, the recording process was then stopped. From the data 209 collected, the position with the longest time duration was 210 selected as the wrist orientation for the relax position/stop 211 position.

213
The wheelchair is capable of five maneuvers, which are 214 moving right, left, forward, backward, and stop. Therefore 215 using only the wrist position as the input variable, the system 216 requires five positions to represent the five buttons available 217 on the GUI. Two control algorithms were developed in this 218 research. They were the threshold value-based method and 219 the naïve Bayesian method. Which: 239 P th , Y th and R th = threshold pitch, yaw and roll. 240 P n , Y n a nd R n = normal for pitch, yaw and roll.

241
SD P , SD Y andSD R = standard deviation for pitch, yaw and 242 roll.   classify the hand gesture is illustrated in Fig. 8.
The probability of the variables in a gesture (P(X|C)) is 262 calculated using Eq. (7).     The result showed that the best performance was in the  Based on the establishment of the stop hand position, four 297 orientations were still required to control the wheelchair, 298 including the forward, backward, right, and left maneuvers. 299 Figure 10(a) and (b) are the positions for forward and back-300 ward maneuvers, having a clockwise and counter-clockwise 301 rotation along the x-axis, respectively. The right and left turns 302 of the wheelchair were further derived from the hand rota-303 tions along the z-axis. In addition, the clockwise and counter-304 clockwise rotations along the z-axis are shown for the right 305 and left turns in Fig. 11(c) and(d), respectively.

306
Further experiments were conducted to determine the pat-307 tern of pitch, yaw, and roll signals when the hand gesture 308 relatively changed to the stop position. Figure 12 shows these 309 variables when the hand gesture changes from the stop to the 310 right. The stop maneuver started from 3 to 6 secs and was 311 marked by t1 period. After 6 secs, it went to t2, as the hand 312 rotated along the z-axis to perform the right gesture.

313
When the variable value after the movement was bigger 314 and smaller than normal, positive (+) and negative (-) signs 315 were indicated, respectively. The normal value was calculated 316 based on the data collected from the volunteers. For this 317 method, the complete table for each hand gesture is shown in 318 Table 2. It can be observed that the forward and left maneu-319 vers had similar signs, indicating a challenge to determine the 320 motion only with this information.

321
The characteristics of all variables for the four hand ges-322 tures were analyzed, with the volunteers performing each 323 action 18 times. The results are shown in Table 3. More-324 over, the pitch was averagely negative for all gestures, as the 325 hand orientation was downward due to gravity. This direc-326 tion reduced the effort to withstand the gravity force. Also, 327 almost all gestures had positive yaw, except the left maneuver, 328 indicating that the hand position was identical to the normal 329       to being the only condition with a negative sign for the pitch. 337 The algorithm will next check the right gesture when the 338 signal did not meet the backward condition. Besides the stop, 339 only three possible gestures were observed when the maneu-340 ver was not in a backward condition. Among these three, the 341 right maneuver was the only gesture with a positive change 342 in the yaw value. For that reason, this variable is used to 343 detect the right gesture. The forward and left gestures were 344 then distinguished based on the pitch value. According to 345 the experiment, the average pitches for the forward and left 346 gestures were positive and negative, respectively. The gesture 347 was assumed as the stop position when all conditions were not 348 met. The algorithm based on the threshold method is shown 349 in Fig. 13.

350
The confusion matrix of the threshold method is given 351 in Table 4. The accuracy of the threshold method is also 352 investigated using the data collected from 40 participants. 353  As participants were ordered to perform each gesture 354 18 times, there were 720 data for each gesture, indicating a 355 total of 3600 motions.

356
The performance of the threshold method is shown in 357 Table 5. Generally, the threshold method has high accuracy  The accuracy, however is unbalanced since the TNR is 98% The naïve Bayes method is implemented to improve the 374 accuracy of the system. Table 6 shows the confusion matrix 375 of Naïve Bayes. Firstly, the method was evaluated with the 376 same data utilized for the threshold approach. This was the 377 training data to determine the mean and standard deviation of 378 each gesture variable. The trained naïve Bayes parameter was 379 then evaluated on a separate testing set. The testing set had a 380 similar amount with the training set.

381
The training performance of Naïve Bayes method is shown 382 in Table 7. Naïve Bayes method has a higher performance 383 than threshold method with 99% accuracy. Both recall and 384 selectivity indicators show that Naïve Bayes method could 385 detect not only the desired gesture but also a movement 386 not in accordance with a gesture. Naïve Bayes method also 387 has higher precision than the threshold method, with 99% 388 precision on average. The result shows that TNR variable 389 of all gestures has 100% performance. This is outstand-390 ing because it indicates that the method could detect unde-391 sired gestures very well. The period of changing the hand 392 position in the dynamic hand gesture can be one of the 393 causes for errors to occur [47]. The shaded area, in Fig. 12, 394 is an example of a transition period from one state to 395 another.

396
The method also has no requirement for a participant 397 to calibrate their hand position. Based on the results, the 398 naïve Bayes has a better performance than the threshold 399 based method. The accuracy of Naïve Bayes also produces a 400 good performance on the testing set. The average accuracy 401 and precision are found to be 99% and 98%, respectively 402 in testing set. Following these findings, the Naïve Bayes 403 method is used as the gesture recognition method for the 404 control system of the wheelchair for the experiment in this 405 study.

406
From the literature, there were several methods of hand 407 gesture recognition have been carried out by researchers, 408 as shown in Table 8     difficulty lifting their shoulders and elbows because of their 431 disability. The designed hand gestures are simple movements 432 that are easily remembered and formed to make the user 433 comfortable.

434
Human robot interaction is the main goal of developing the 435 hand gesture recognition [12]. So, the qualitative testing of 436 the proposed system was conducted to test the wheelchair. 437 Figure 14(a) illustrates the user operating the wheelchair 438 while the right hand performs the normal gesture for the 439 stop command and Fig. 14