Detection of the Intention to Grasp During Reaching in Stroke Using Inertial Sensing

To support stroke survivors in activities of daily living, wearable soft-robotic gloves are being developed. An essential feature for use in daily life is detection of movement intent to trigger actuation without substantial delays. To increase efficacy, the intention to grasp should be detected as soon as possible, while other movements are not detected instead. Therefore, the possibilities to classify reach and grasp movements of stroke survivors, and to detect the intention of grasp movements, were investigated using inertial sensing. Hand and wrist movements of 10 stroke survivors were analyzed during reach and grasp movements using inertial sensing and a Support Vector Machine classifier. The highest mean accuracies of 96.8% and 83.3% were achieved for single- and multi-user classification respectively. Accuracies up to 90% were achieved when using 80% of the movement length, or even only 50% of the movement length after choosing the optimal kernel per person. This would allow for an earlier detection of 300–750ms, but at the expense of accuracy. In conclusion, inertial sensing combined with the Support Vector Machine classifier is a promising method for actuation of grasp-supporting devices to aid stroke survivors in activities of daily living. Online implementation should be investigated in future research.

of the upper extremities [2]. The impairments manifest as muscle weakness, changes in muscle tone and a decrease in motor control, which results into difficulty of successfully performing basic hand movements such as reaching, grasping objects, and holding objects [3]. Even though strategies in rehabilitation therapy exist to aid recovery, 40% of the total stroke population suffers from chronic motor impairment [3], [4]. Stroke survivors with only one functional hand are restricted in the performance of daily life tasks like preparing meals, housework, and shopping [4], [5].
Stroke survivors can be supported by means of robotic orthoses, functional aids, casts, splints, biofeedback, and electrical stimulation [6], [7]. However, most of these technologies either are bulky, restrict movement or are uncomfortable [7]. The development of orthoses to support the hand during activities of daily living (ADL) is becoming increasingly prevalent [8]. The majority of them is controlled by either surface electromyography, interaction forces or human movement [8]. One developed orthosis that is slim, portable, and developed to support grip strength in ADL, is the Soft Extra Muscle (SEM) Glove™ from Bioservo Technologies AB [9] Force sensors measure forces during grasping, which are located at the gray circles of the distal phalanges of the thumb, middle-and ring finger when worn (Fig. 1a). The SEM Glove™ is developed to support users in grasping and holding objects by actuation of artificial tendons to apply extra force to the object as soon as a grasping force is detected. This could make the SEM Glove™ a suitable solution for stroke survivors that do not show spasticity or contractures but still experience problems with executing grasping movements. Up to now, a comparable, further developed, version of the SEM Glove™ is the only wearable soft-robotic glove that has shown to be feasible when used for four weeks independently during ADL at home by stroke patients [10].
The force sensors on this glove are currently the only source of detection of grasp movements, which means that detection occurs after the subject touches the object. In a study performed by Radder et al. [11], elderly people with declined hand function, performed tasks considerably faster without a comparable glove having the same force sensors compared with performance of the same task with glove. To increase efficacy of the SEM Glove™ to support stroke survivors in ADL, the intention to grasp should be detected as soon as possible while other movements, such as reaching, are distinguished from grasp movements.
The possibility to detect different final hand postures of both healthy subjects and stroke survivors was already investigated This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see http://creativecommons.org/licenses/by/4.0/ by several studies [12]- [19]. To classify the final hand posture, bend sensors, pressure sensors, position of the fingertips, electromyography, electromagnetic sensors in combination with electromyography, and segment angles were used. In contrast to classifying different final hand postures, de Vries et al. [20] studied the possibility to distinguish reach from grasp movements while using a minimal amount of inertial sensing. In that study reach and grasp movements were classified in healthy subjects using single-and multi-user classification support vector machine (SVM) classifiers with accuracies up to 98.2% and 91.4% respectively. By using 40% of the available data of a single movement, an accuracy of 85.3% was achieved. Although at the expense of accuracy, grasp movements could be detected up to 1200 ms before the subject touches the object. However, stroke survivors show altered upper limb kinematic movement characteristics within a reach and grasp movement as compared to healthy subjects [21]. Therefore, as a next step, this research needs to be translated such that ultimately the prospective solution could be applied in the gripsupporting glove for stroke if proven useful. Hence, the goal of this study was to investigate the possibilities to classify reach and grasp movements of stroke survivors by analyzing their finger and hand movements using a minimal number of inertial sensors. Furthermore, the possibilities to detect a grasp movement by analyzing the intention of the movement was investigated.

A. Participants
Ten stroke survivors were recruited for this cross-sectional study performed at Roessingh Research and Development (RRD), Enschede, the Netherlands. Criteria for inclusion into this study were: 1) clinically diagnosed with unilateral ischemic or hemorrhagic stroke at least three months before inclusion; 2) between 18-80 years of age; 3) able to actively extend the fingers enough to grasp a cylindrical object with a diameter of 6 cm; 4) able to actively extend the fingers enough to grasp a ball with a diameter of 7.5 cm 5) a sufficient cognitive status to understand two-step instructions in Dutch; and 6) having (corrected to) normal vision. Criteria for exclusion were: 1) severe sensory problems or pain of the affected hand; 2) severe contractures limiting the passive range of motion and; 3) co-morbidities limiting functional use of the hand. The Medical Research Ethics Committee (MREC) Twente, the Netherlands, approved the study (CCMO number NL64511.044.17). All stroke survivors provided written informed consent prior to the start of this study.

B. Instrumentation
The force sensors in the three-fingered SEM Glove™ were used to detect, and determine the duration of, contact with the object in the case of a grasping movement. The thumb, middleand ring finger are covered by the glove. The force sensors are located at the distal part of those fingers. Relaxation of active finger flexion enables releasing of the object. An inertial measurement system [22] with inertial measurement units was placed on the ulnar styloid, the dorsal side of the hand, and the  phalanges of the thumb, index and middle finger to measure the angular velocities in order to detect flexion and extension movements of the thumb, fingers and wrist. A combination of the SEM Glove™ and the inertial measurement system as used in the current study can be seen in Fig.1.

C. Experimental Set-Up
The set-up used in this study ( Fig. 2) was similar to study design of de Vries et al. [20]. The participant was seated at a table with adjustable height to make sure that the elbow of the affected side was flexed 90 • and aligned with the trunk. Targets were presented in five horizontal directions (0 • , 45 • , 90 • , 135 • , and 180 • ) on locations within the active reaching range of motion of the participant. The affected hand was initially positioned in the middle of the semi-circle.

D. Protocol
Prior to the start of the reach and grasp movements, the upper extremity part of the Fugl-Meyer assessment was performed to evaluate the motor status and degree of synergies in the upper limb [23]. Thereafter, anatomical rotation axes were defined with a sensor-to-segment calibration, described in the section below.
As described by de Vries et al. [20], two different grasp tasks and two different reach tasks towards the five different locations were performed: 1) grasping a wooden ball with a diameter of 7.5 cm; 2) grasping a cylindrical object with a diameter of 6 cm; 3) reaching towards a target location whilst in pronation; and 4) reaching towards a target location whilst in supination. The grasp gestures, cylindrical and spherical, represent two common grasps in stroke [24]. The protocol allowed compensatory movements to ensure that natural movements from each individual were captured. In healthy subjects, a minimum number of 70 movements were needed to train the classifier accurately [20]. Based on the results from de Vries et al. [20], it was decided to repeat all tasks 5 times per location. Besides a specific task and location, the patient was also instructed to have one of two different starting hand postures; either flat on the table with the dorsal side upwards or making a fist with the medial side resting on the table. Therefore, a total of 100 grasp and 100 reach movements were performed of which the order of the tasks as well as the order of initial hand posture and the locations were randomized.

E. Sensor-to-Segment Calibration
A sensor-to-segment calibration as described by Luinge et al. [25] was performed to determine the anatomical rotation axes. Participants were instructed to stand upright and hold their elbow in a flexion angle of 90 • while the dorsal side of the hand faced upwards as reference position. Participants were then asked to perform and repeat six tasks five times: 1) flexion and extension of the fingers; 2) ab-and adduction of the fingers; 3) flexion and extension of the thumb; 4) flexion and extension of the wrist; 5) ab-and adduction of the wrist; 6) flexion and extension of the elbow. The coordinate system of each segment was defined according to the coordinate system of the whole body in anatomical position. The x-axis was defined as the anteroposterior axis pointing in anterior direction representing ab-and adduction, the y-axis was defined as the mediolateral axis pointing in lateral direction of the right hand representing flexion and extension, and the z-axis was the longitudinal axis pointing in caudal direction, which represents pro-and supination. The direction of the x-axis was determined by measuring the gravitational force at the reference position. By analyzing the direction of angular velocity during flexion of the different segments (middle finger, index finger, thumb, wrist and arm), the direction of the y-axis was determined. The direction of the z-axis was then determined by the cross-product of the x-and y-axis. To correct for unwanted movements performed in the reference position and to make the coordinate system orthogonal, the direction of the x-axis was recalculated by computing the cross-product of the y-and z-axis. Finally, to acquire the segment data with the coordinate system of the human body, the gyroscope data was multiplied with the rotation matrices specified by the unit vectors.

F. Data Analysis
Data was acquired from the SEM Glove™ using Tera Term version 4.98 and from the inertial measurement system using MATLAB version 2016b on a laptop running a 64-bit Windows 10 OS with a 2.20 GHz i7-2670QM Intel® Core™ CPU and 6 GB of RAM. For safety reasons, the SEM Glove™ was connected to the laptop with a USB isolator (Model UH401) from Advantech. Data analysis was performed using MATLAB version 2017b on a desktop running a 64-bit Windows 10 OS with a 3.40 GHz i5-7500 Intel® Core™ CPU and 8 GB of RAM.
1) Pre-Processing: The offset of the sensors from the inertial measurement system and the SEM Glove™ was removed from a manually determined baseline at the beginning of each recording. Throughout the baseline, no movement of the hand and no force on the SEM Glove™ was present. After removal of the offset, data from the inertial measurement system was filtered with a 4 th order, zero-lag, low pass Butterworth filter. The cut-off frequency was set at 6 Hz. [26], [27] 2) Training and Classification: Within the classification learner toolbox of MATLAB, the Support Vector Machine (SVM) was used for classification of the data since it is able to find patterns in high dimensional, non-linearly separable data and can accurately distinguish between two discrete classes [28], [29]. The two defined classes in the current study were 'reach' and 'grasp' and the SVM classifiers were trained and validated using 10-fold cross-validation. Besides the standard linear boundary, the quadratic, cubic, fine Gaussian, medium Gaussian and coarse Gaussian kernels were investigated in the current study. Classification was performed in two scenarios: 1) by splitting the dataset of one patient into a test and training set, i.e. single-user analysis; 2) by splitting the dataset of all patients into a test set of one patient and a training set of other patients, i.e. multi-user analysis.
3) Database: The data were divided into 200 trials per subject, each trial containing a reach or grasp movement. During a reach movement, the beginning and end of a trial were determined using a threshold detector algorithm for the angular velocity (threshold ± 0.1 rad/s) [20]. In case of a grasp movement, the end of a trial is defined as the moment the force sensors in the SEM Glove™ detected contact. 4) Single-User: In case of single-user classification, 25% of the 200 trials (25 grasp and 25 reach trials) was randomly selected for the test set. The remaining 150 trials of the same participant were used for training. 5) Multi-User: Two methods of multi-user classification were performed. First, the trials of one participant were selected for the test set, while the trials of the remaining subjects were selected for the training set. Second, the participants were first divided into categories of stroke severity (mild, moderate and severe [23]) following the FMA score after which the first classification method was done for each group. For each participant, a separate database was created.
6) Feature Extraction: The extraction of features was performed in a similar way as by de Vries et al. [20]. The movements of each segment were expressed with respect to the dorsal side of the hand by determining the relative angular velocity of a segment s (ω s ) in each rotation axis separately and the norm angular velocity vector of a segment s (norm s ). The previously described sensor-to-segment calibration was used to determine the relative angular velocity, ω s , by subtracting the data of the dorsal side of the hand from the data of a segment s. The second parameter, norm s , was calculated by taking the norm of the difference in angular velocities on the x-, y-, and z-axes between a segment s and the dorsal side of the hand. Two features of the segments, the mean and standard deviation (SD), were calculated for both parameters to get eight features per segment relative to the dorsal side of the hand; two for the norm and six for the three components of the relative angular velocity. In total four segments were used to calculate the separate features: the distal part of the forearm and the distal phalanges of the thumb, index and middle finger. So in total, a number of 32 features were extracted from each trial.
Several different combinations of these features were determined to train and test the SVM classifier. To limit the number of sensors, at most two sensors were used in each feature combination: the dorsal side of the hand with one other segment. Table 1 shows the combinations of features used in the experiment. The first four feature combinations consist of the mean and SD of the norm angular velocity vector of the middle finger, index finger, forearm and thumb with respect to the dorsal side of the hand. Feature combinations 5 to 8 represent the mean and SD of the relative angular velocities of all axes of the described segments. Because the y-axis of each segment represents flexion and extension, the mean and SD of the relative angular velocity of only the y-axis were used as separate feature combinations, which are numbers 9 to 12. Finally, due to the saddle joint of the thumb, the mean and SD of the relative angular velocities of the x-and z-axis of the thumb were also used as the last two feature combinations. This means that all feature combinations consist of two features per combination, except for combinations 5 to 8, which contain six features each.

A. Participants
Ten chronic stroke survivors were included in this study ( Table 2). Based on a categorization of the FMA score without reflexes [30], six (60%) of the included stroke survivors were mildly affected (FMA score > 41), and four (40%) were moderately affected (28 ≥ FMA score ≥ 41). In six

B. Single-User Classification
Using single-user classification for all combinations of feature combination and kernel, which required on average a computing time of 175.0s (±40.4s), the highest mean accuracy of 96.8% (±4.54%) was achieved by the mean and SD of the relative angular velocities of all axes of the middle finger (feature combination 5) with the medium Gaussian kernel (Table 3). This combination of the SVM classifier was trained in 0.073s on average and showed a mean sensitivity and specificity of 98.0% (±3.37%) and 96.1% (±6.25%) respectively. When optimizing the feature combination and kernel for each person, accuracies ranging from 96.0%-100% were found, with an average accuracy of 99.0%.

C. Multi-User Classification
The highest mean accuracy of 83.3% (± 9.99%) for the multi-user classification for all combinations of feature combination and kernel was achieved by the mean and SD of the relative angular velocities of all axes of the middle finger (feature combination 5) with the medium Gaussian kernel (Table 4). This combination of the SVM classifier was trained in 0.43s on average and showed a mean sensitivity and specificity of 87.2% (±8.22%) and 83.0% (±12.8%) respectively. The highest mean accuracy after the categorical multi-user classification for the mildly affected stroke survivors, 85.3% (±8.31%), was achieved by the mean and SD of the relative angular velocities of all axes of the middle finger (feature combination 5) with the linear kernel and was trained in 0.5s on average. For the moderately affected category, the highest accuracy was 77.4% (±12.7%) by the mean and SD of the relative angular velocity of the y-axis of the index finger (feature combination 10) with the coarse Gaussian kernel and was trained in 0.12s on average.

D. Grasp Intention Detection
The mean accuracies and SD using the described variation of trial lengths for the single-user classification are shown in Table 5. Using 80% of the movement length, a mean accuracy of 87.6% (±9.42%) was achieved by taking the mean and  FEATURE COMBINATIONS CORRESPOND TO THE  DESCRIBED FEATURE COMBINATIONS IN TABLE 1. THE KERNEL WITH THE HIGHEST ACCURACY FOR EACH FEATURE  COMBINATION IS MARKED IN BOLD TEXT AND THE HIGHEST ACCURACY OVERALL IS UNDERLINED   SD of the relative angular velocities of all axes of the middle finger with a medium Gaussian kernel. After optimization of the kernel per person, an accuracy of 90.4% (±6.10%) was achieved when using only 50% of the movement length. The intention to grasp could be detected approximately 750 ms earlier with the IMU and SEM Glove™ setup combined with the SVM classifier than the SEM Glove™ using the force sensors when using only 50% of the movement length.

IV. DISCUSSION
The current study explored possibilities to classify reach and grasp movements, as well as to detect the intention to grasp, of stroke survivors by analyzing their finger, hand and wrist movements using a minimal number of inertial sensors. From the results of the experiment, it can be said that reach movements can be distinguished from grasp movements by using only two IMUs: one sensor on the dorsal side of the hand and one on a distal phalange of the thumb, middleor index finger or distal part of the forearm. By using the single-user classification method, the highest mean accuracy of 96.8% was achieved whereas the multi-user classification method achieved a highest mean accuracy of 83.3%. In both cases these highest accuracies were achieved by the mean and SD of the relative angular velocities of all axes of the middle finger (feature combination 5) with the medium Gaussian kernel. After optimizing the feature combination and kernel per person, accuracies ranging from 96% to 100% were reached. Accuracies up to 90% were achieved when using 80% of the movement length by taking the mean and SD of the relative angular velocities of all axes of the middle finger with a medium Gaussian kernel, or even only 50% of the movement length after optimizing the kernel per person. This would allow for an earlier grasp detection of 300 ms (80% movement length) to 750 ms (50% movement length) compared to grasp detection by the force sensors in the original setup of the SEM Glove™.
So far, studies distinguishing different grasps in stroke show comparable or lower accuracies (87% [18], 89.5% [17] and 73% [19]), using segment angles [18] or EMG signals [17], [19] for classification. In contrast to those studies, the present study used a minimal number of two inertial sensors and distinguished between reach and grasp, instead of using multiple sensors and distinguishing between different grasps. In the work of de Vries et al. [20], single-and multiuser classification accuracies of 98.2% and 91.4% respectively were achieved when distinguishing reach from grasp in healthy participants. In the case of single-user classification, our result is similar to the results of the De Vries study, but the multi-user classification showed a substantial lower accuracy. As previously described, stroke survivors show different and more variable movement patterns as compared to healthy controls (21). Additionally, stroke survivors usually experience spasticity which affects their ability to extend their fingers. This all can explain that a multi-user classification method for stroke survivors is not as robust as for healthy subjects with less variable movement patterns.
Moderately affected stroke patients generally have a larger movement variability than mildly affected stroke patients [21]. When dividing the stroke participants from the present study in two categories for the multi-user classification and comparing the mean accuracies to the first grouped multi-user classification, comparable results for the mild category were achieved, but worse results were found for the moderate category. A similar trend, in which lower accuracies occurred when subjects were more severely impaired, was also found in the study of Lee et al. [19]. Altogether, for the SVM classifier to be feasible to distinguish reach from grasp of stroke survivors accurately, a single-user classification method might be the better option. Because optimizing the feature combination and kernel per person showed possible increases of the accuracy up to 100%, for future developments a tailormade system, in terms of software, should be considered. Concerning computation time, this should not propose any problems.
The results from the user dependent grasp intention detection showed that the intention to grasp could be detected 300 ms (80% movement length) to 750 ms (50% movement length) earlier than when using the SEM Glove™ in its original configuration using force sensors. Depending on the accuracy that is desired, grasp intention could be detected even earlier than 750 ms before the SEM Glove™ currently does by optimizing sensitivity and specificity trade-offs individually. If a stroke survivor, for example, performs solely grasp movements with the affected hand, the system can be more sensitive, but if the movements involve a high variety in reach and grasp, the system could be set at a higher specificity.
This research showed promising results for both single-and multi-user classification to distinguish reach from grasp movements in chronic stroke survivors. The IMU setup used in the present study is suitably small to integrate in the glove itself, if desired. Although the multi-user classification achieved lower accuracies than single-user classification, higher accuracies for a specific stroke survivor could be achieved by adjusting the sensitivity and specificity in a multi-user setting as well. Only two sensors were used for each separate analysis in this experiment. Although from the perspective of cost and robustness it would be preferred to incorporate no more than two sensors in the SEM Glove™, it might be that higher multi-user classification accuracies could be achieved with combinations that use more than two sensors. In the future, combinations of sensors/features such as the mean and SD of the relative angular velocity of all axes of the thumb and middle finger, could be included in the analysis using SVM to determine whether this yields generalizable results. Although adding complexity to the system, this solution might be highly advantageous when applied as intention detection method in assistive technologies, since there is no need for establishing a dataset for training the algorithm prior to use by a subject, and the system can be used in a plug-and-play manner.
All filtering and classification was performed offline in the current study, but ultimately the system should be able to function online before implementing it in the SEM Glove™. When filtering data in real-time, a latency is introduced which should be accounted for during classification. However, this latency can be minimized by choosing a suitable filter and window. Ultimately, it might be beneficial for stroke survivors to not only distinguish between reach and grasp movements, but also between different grasp movements. If the system should be able to classify between different reach and different grasp movements online using the SVM classifier, computational complexity increases and thus detection time will increase and detection accuracies will possibly decrease. Methods of using the SVM online have been proposed [31], but other analysis methods such as a Matched Filter (MF) with a threshold detection algorithm or a finite state model such as a Hidden Markov Model (HMM) [32] might be more suitable in terms of computational complexity. If the SVM classifier were to be used online, it needs to wait for the whole data sequence before doing a classification, whereas a HMM is able to update its prediction every time a part of the sequence is presented and would mean that computational load is lower [33]. A MF algorithm would only involve simplistic calculations which are easily computed. Therefore, the most appropriate method for online implementation while taking into account the computational complexity and its properties and performance for this specific application should be investigated.

V. CONCLUSIONS
In this study, grasp movements of stroke survivors could be accurately distinguished from reach movements using a minimal number of inertial sensors. Promising results for both single-user (96.8%) and multi-user classification (83.3%) were achieved. While using only part of the movement length, accurate grasp detection would allow for a faster grasp detection than the current method for detection in the glove. When in future research comparable results could be achieved after online implementation, inertial measurement units could be used to control devices that aid in daily life activities that involve grasp movements.