Prediction of Dexterous Finger Forces With Forearm Rotation Using Motoneuron Discharges

Motor unit (MU) discharge information obtained via electromyogram (EMG) decomposition can be used to decode dexterous multi-finger movement intention for neural-machine interfaces (NMI). However, the variation of the motor unit action potential (MUAP) shape resulted from forearm rotation leads to the decreased performance of EMG decomposition, especially under the real-time condition and then the degradation of motion decoding accuracy. The object of this study was to develop a method to realize the accurate extraction of MU discharge information across forearm pronated/supinated positions in the real-time condition for dexterous multi-finger force prediction. The FastICA-based EMG decomposition technique was used and the proposed method obtained multiple separation vectors for each MU at different forearm positions in the initialization phase. Under the real-time condition, the MU discharge information was extracted adaptively using the separation vector extracted at the nearest forearm position. As comparison, the previous method that utilized a single constant separation vector to extract MU discharges across forearm positions and the conventional method that utilized the EMG amplitude information were also performed. The results showed that the proposed method obtained a significantly better performance compared with the other two methods, manifested in a larger coefficient of determination ( ${R}^{{2}}\text {)}$ and a smaller root mean squared error (RMSE) between the predicted and recorded force. Our results demonstrated the feasibility and the effectiveness of the proposed method to extract MU discharge information during forearm rotation for dexterous force prediction under the real-time conditions. Further development of the proposed method could potentially promote the application of the EMG decomposition technique for continuous dexterous motion decoding in a realistic NMI application scenario.

SECTION I. Introduction

The hands carry out numerous significant daily tasks and is crucial to our ability to live a normal and independent life. Neural-machine interfaces (NMI) can decode movement intention from biological signals and use them to control rehabilitation devices, prosthetics [1], and exoskeletons [2], which improves the quality of life for people with hand dysfunction. As one of the commonly used physiological signals, surface electromyography (sEMG) can be captured by electrodes affixed to the skin surface, which is non-invasive, secure, and easy to operate [3]. sEMG can be used to perform neuromuscular characterization [4] and is generally employed in motion decoding using the pattern recognition-based method or the proportional control-based method. In the pattern recognition process, a feature extraction stage is used to increase the information density of the EMG signals [5] and classifiers are used to identify the preset motion categories. Typically, characteristics in the time and frequency domain are extracted [6], [7], [8]. The proportional control is more applicable when controlling individual degrees of freedom in a continuous manner is needed.

The sEMG is composed of superimposed motor unit action potentials (MUAP) generated from the recruited motor units (MU). There might be several limitations of sEMG global features for accurate motion decoding, such as the noise introduced by conduction and the signal acquisition procedures [9], signal crosstalk generated from adjacent muscles or muscle compartments [10], signal amplitude cancellation caused by the superimposition of MUAP waveforms [11]. Unlike the EMG global features, MU discharge activities directly reflect the neural commands sent by the central nervous system in binary codes [12], and the MU discharge information can be extracted through the EMG decomposition techniques including the template matching [13], the blind source separation (BSS) [14], [15], and the machine learning [16] methods. The neural drive information that can be estimated as the compound firing rate at the group level of MUs has been demonstrated to be superior to the conventional global EMG features in hand gesture recognition [17], [18] and continuous estimation of muscle force [19], [20], [21], torque [22], and joint angle [23], [24].

For a realistic NMI system, motion decoding needs to be performed in real-time. Currently, online EMG decomposition techniques typically assume the steadiness of the MUAP profiles. Therefore, the MU feature (e.g., the separation vector in the BSS-based method) to extract the discharge activity can be estimated in the initialization phase and kept constant during online decomposition. However, it has been demonstrated that the MUAP shape can be affected by multiple geometric and anatomical factors such as the muscle fiber length and the electrode location and orientation [25]. Among these, electrode shifts can cause the change of the MUAP shape and have been shown to reduce the classification accuracy and real-time controllability [26], [27]. Forearm rotation (pronation and supination) may be one of the major and unavoidable factors that can change the musculoskeletal geometry [28] and cause the electrodes to shift relative to the muscle [29] in the realistic application scenarios. The variation of the MUAP shape could lead to the decreased detection accuracy of the discharge activity and the performance degradation of motion decoding using the pre-estimated constant MU features. The previous study [30] showed that the neural drive method did not always have better performance than the conventional EMG global features in motion decoding across forearm postures if the MU features were extracted at, to some degree, an arbitrary forearm posture. Although some previous studies proposed to use separate motion decoding models for individual limb positions [31], [32], [33], they did not demonstrate the applicability of the decoding models obtained at some specific limb positions when used at other limb positions. This is crucial because the limb position changes continuously in a realistic scenario, and it is impossible to obtain the decoding models or separation vectors in every successive limb position.

In order to alleviate this issue, a simple framework was proposed in this study to extract MU discharge activities in real-time across forearm positions for dexterous multi-finger force prediction. The proposed method utilized the BSS-based EMG decomposition technique and assumed that the multiple separation vectors extracted at different forearm positions for each MU can obtain improved EMG decomposition accuracy by using the separation vector extracted at the nearest forearm position. The previous method that utilized a single constant separation vector to extract MU discharges across forearm positions and the conventional method that utilized the EMG amplitude information was also performed as comparison. The three methods were tested on the dexterous force prediction across different forearm positions during an isometric multi-finger extension experiment. Our results showed that the proposed method obtained the best performance (a higher correlation and a smaller prediction error) in force prediction, which demonstrated the feasibility and the effectiveness of the proposed method to extract MU discharge information across forearm positions for dexterous force prediction under the real-time conditions.

SECTION II. Method

A. Experiment

Ten subjects (21-26 years old) without any known hand movement dysfunctions were recruited in this study. All subjects gave informed consent with protocols approved by the Ethics Committee in Xi’an Jiaotong University.

Subjects were seated at a chair adjusted to the appropriate height. The wrist was fixed into an angle-adjustable device, with the elbow supported to keep the arm in a horizontal position (Fig. 1(a)). The index, middle, ring, and pinky fingers were tapped on four load cells (DJSX Pressure Sensor, Shanghai Di Jia, China) separately to record finger extension forces. The wrists and palms were restrained from both the anterior and posterior positions to prevent forces from the wrist onto the load cells.

Monopolar high-density EMG signals were recorded with the $8\times 16$ electrode array placed on the extensor digitorum communis (EDC) muscle (Fig. 1(a)). The diameter of each electrode is 3 mm and the distance between the adjacent electrodes is 10 mm. EMG signals were sampled at 2048 Hz and amplified with a gain of 150 and a pass band of 10–500Hz (Quattrocento, OT Bioelettronica). The reference and the common ground were located at the elbow and the wrist, respectively. The extension force of individual fingers was recorded synchronously at 1000Hz.

The experiments were conducted at five forearm positions from pronation to supination, defined as 30°, 60°, 90° (neutral position), 120° and 150° in this study (Fig. 1 (b)). At each forearm position, subjects were asked to extend four fingers alternatively with the maximal effort to measure maximum voluntary contraction (MVC) at the beginning of the experiment. Due to the relatively large enslaving effect between the ring and pinky fingers [34], [35], subjects were requested to extend the ring and pinky fingers (termed the ring-pinky finger) together and their extension forces were always summed up for further analysis.

Fig. 1.

Experimental setup. (a) High-density EMG signals were recorded with an ${8}\times {16}$ electrode array covering the extensor digitorum communis muscles and extension forces were recorded with four load cells. Forearm pronation and supination positions were adjusted using the angle-adjustable device. (b) The five forearm positions at which subjects performed finger extensions. (c) The trapezoidal force target of a trial.

At each forearm position, subjects were asked to perform two types of tasks, i.e., the extension of a single finger (single-finger task) or multiple fingers (multi-finger task) to follow the target force trajectory as shown in Fig. 1(c). The force trajectory for the single-finger and multi-finger trials were the same and consisted of 9 pseudorandom trapezoids with a total of 180-second length. In the single-finger trials, subjects were asked to extend one of the three (index, middle and ring-pinky) fingers throughout the whole trial and co-contractions of other fingers should be limited to the minimum. In the multi-finger trials, the index, middle, and ring-pinky fingers extended alternately in a trial (Fig. 1(c)) with a random order instructed from the experimenter. At any moment, the subjects need to control the extension force of the target finger to follow the force trajectory while the other two non-target fingers can be extended at will. Only one target finger was assigned at any moment because it would be hard for subjects to follow three force trajectories for three fingers simultaneously. At each forearm position, three single-finger trials with each for a finger and one multi-finger trial were performed, resulting in 15 single-finger trials and 5 multi-finger trials in total for each subject across 5 forearm positions.

B. Force Prediction Using Neural Drive Information

Force prediction using the neural drive information included two procedures, the offline initialization procedure, and the online force prediction procedure. In the offline initialization phase (Fig. 2), the EMG signals from the single-finger trials were used to obtain the MU features and the force prediction model for individual fingers. In the online force prediction phase, the multi-finger trials were used to validate the performance of predicting multi-finger forces concurrently.

1) Offline EMG Decomposition

The Fast Independent Component Analysis (FastICA) algorithm was used to perform EMG decomposition in this study [14], [36]. The raw EMG signal ${\mathrm {\bf x}}=[x_{1} (t),x_{2} (t),\cdots ,x_{m} (t)]^{\textrm {T}}$ , $t=1$ ,2,..., $L$ with $m$ as the number of EMG electrodes was extended by adding $R$ ($R=7$ in this study) delayed versions of each channel and whitened, resulting in the Rm$\times L$ matrix z. Then, 200 loops of iteration in the FastICA algorithm were performed on z, and each loop converged to the feature of one MU including its separation vector w, the source signal s and the spike train t. The source signal can be obtained as

$$\begin{equation*}{\mathrm {\bf s=w}}^{\textrm {T}}{\mathrm {\bf z}} \tag {1}\end{equation*}$$ and the spike train was obtained through the binary classification of the peaks in the source signal using the Kmeans++ algorithm [37]. Then, the Silhouette (SIL) value was calculated to quantify the distance between two classes of source peaks and the MUs with a SIL value smaller than 0.6 and duplicates of other MUs were removed from further analysis.

After the offline EMG decomposition was performed on all single-finger trials, several MUs with their separation vectors and spike trains were obtained for individual fingers at each forearm position (Fig. 2(a)). The feature of the $i$ th MU for finger $j$ at position $k$ was termed as $\{{\mathrm {\bf w}}_{i}^{j,k},{\mathrm {\bf t}}_{i}^{j,k} \}$ with $i=1$ ,2,..., $j\in$ {index, middle, ring-pinky} and $k\in$ {30°, 60°, 90°, 120°, 150°}. At this stage, $j$ was determined as the finger to perform the corresponding single-finger trial. The average numbers of MUs obtained after offline decomposition across subjects are shown in Table I.

TABLE I. Average Number of MUs

2) MU Pool Refinement

Although subjects were requested to extend only one finger in the single-finger trials, the co-activation of other fingers cannot be completely avoided. This led to the fact that the MUs obtained in the single-finger trials may not exclusively belong to the specific finger. Therefore, a MU classification procedure was used to refine the MU pool for individual fingers (Fig. 2(b)) and has been demonstrated to improve the estimation performance of multi-finger forces in the previous study [38]. Specifically, for a single-finger trial, the time courses of the firing rate were estimated using a sliding window with a 0.5-second length and a 0.2-second step. In the subsequent text, the sliding windows are the same if not specified. The forces of the three fingers were also processed using the sliding window. Then, the Pearson correlation coefficient was calculated between the firing rate of all MUs and the force of all fingers, resulting in the value $r{ }_{i,p}^{j,k}$ representing the correlation coefficient between the firing rate of the $i$ th MU and the force of finger $p$ from the single-finger trial of finger $j$ at the forearm position $k$ . The $i$ th MU was removed from the pool $\{{\mathrm {\bf w}}_{i}^{j,k} \}$ of finger $j$ if $r{ }_{i,p}^{j,k} \gt r{}_{i,j}^{j,k}$ with $p\ne j$ existed. The average numbers of MUs removed in the refinement procedure across subjects are shown in Table I.

3) Multiple Separation Vectors for One MU

Due to the variation of the MUAP shape across forearm positions, the separation vectors of the same MU might be markedly distinct at different positions. This might result in the inaccurate detection of spikes if a single constant separation vector is used across forearm positions. In order to solve this issue, a multi-separation vector strategy was proposed in this study. Specifically, for each MU, multiple separation vectors at different forearm positions were extracted, and the one extracted at the nearest forearm position was always used during online decomposition (Fig. 2(c) and 2(d)).

Fig. 2.

Flow chart of the NRL-mulSV and the NRL-sinSV methods to predict finger (for example, index) forces with forearm rotation. (a) Offline EMG decomposition on all single-finger trials using the FastICA algorithm. (b) MU pool refinement for individual fingers according to the correlation between MU firing rates and finger forces. Only partial representative MUs are shown to save space. (c) The collection of separation vectors for a specific MU across forearm positions according to the spike consistency values. (d) The multiple separation vector strategy proposed in this study and the previous single separation vector strategy as the control. (e) A multivariate linear regression model for individual fingers to map MU firing rates to finger forces.

The same MU can be captured via offline decomposition multiple times from different forearm positions. In order to collect the separation vectors for a specific MU, the measurement of spike consistency (SC) between two spike trains was used in this study and calculated as:

$$\begin{equation*} SC={2n_{\textrm {syn}}} \mathord {\left /{{\vphantom {{2n_{\textrm {syn}}} {(n_{1} +n_{2} )}}}}\right . \hspace {-1.2pt} } {(n_{1} +n_{2} )}\times 100\% \tag {2}\end{equation*}$$ where $n_{\mathrm {syn}}$ was the number of synchronized spikes between two spike trains, and $n_{1}$ and $n_{2}$ were the number of spikes in the two spike trains, respectively.

First, for a given finger $j$ , the EMG signals from all its single-finger trials were concatenated head-to-tail and the MU separation vectors of finger $j$ from all forearm positions were grouped together{${\mathrm {\bf w}}_{1\sim N_{1}}^{j,30^{\circ }},{\mathrm {\bf w}}_{1\sim N_{2} }^{j,60^{\circ }},{\mathrm {\bf w}}_{1\sim N_{3}}^{j,90^{\circ }},{\mathrm {\bf w}}_{1\sim N_{4}}^{j,120^{\circ }},{\mathrm {\bf w}}_{1\sim N_{5}}^{j,150^{\circ }}$ }, with $N_{1}$ , $N_{2}$ , $N_{3}$ , $N_{\mathrm {4}}$ and $N_{5}$ as the number of MUs obtained at the corresponding forearm positions. Second, the resultant 900-second EMG segment was extended and whitened. The source signals and then the spike trains corresponding to individual separation vectors were obtained using (1), resulting in the 900-second spike trains {${\mathrm {\bf t}}_{1\sim N_{1}}^{j,30^{\circ }},{\mathrm {\bf t}}_{1\sim N_{2} }^{j,60^{\circ }},{\mathrm {\bf t}}_{1\sim N_{3}}^{j,90^{\circ }},{\mathrm {\bf t}}_{1\sim N_{4}}^{j,120^{\circ }},{\mathrm {\bf t}}_{1\sim N_{5}}^{j,150^{\circ }}$ }. Third, the SC values were calculated between any pair of spike trains from any two forearm positions, resulting in $SC_{i_{1} i_{2}}^{j,k_{1} k_{2}}$ as the SC value between spike train ${\mathrm {\bf t}}_{i_{1}}^{j,k_{1}}$ and spike train ${\mathrm {\bf t}}_{i_{2}}^{j,k_{2}}$ . The corresponding two separation vectors ${\mathrm {\bf w}}_{i_{1}}^{j,k_{1}}$ and ${\mathrm {\bf w}}_{i_{2}}^{j,k_{2}}$ were considered to belong to the same MU if $SC_{i_{1} i_{2}}^{j,k_{1} k_{2}} \gt 0.7$ . Lastly, the separation vectors from different forearm positions were categorized into different MUs based on the above matching procedure. Since some MUs may have no corresponding separation vectors obtained at some forearm positions after the above matching procedure, the separation vector from the adjacent forearm position with a higher SIL value was selected. In the end, five separation vectors for 5 forearm positions were available for each MU of finger $j$ . In the subsequent text, the neural drive-based force prediction method that adaptively used the separation vector obtained at the nearest forearm position to extract MU discharge activities was termed the ‘NRL-mulSV’ method. The average numbers of MUs used for online force prediction across subjects are shown in Table I.

The previous method [30] that used a single separation vector across forearm positions was also performed as the comparison. Instead of selecting the separation vector from the neutral position for all MUs [30], an optimization procedure was carried out in this study. After the five separation vectors were assigned to individual MUs as described in the NRL-mulSV method, five 900-second spike trains corresponding to 5 separation vectors can be obtained for each MU. The spike trains together with the corresponding force data were processed using the sliding window. Then, a linear regression was performed between the firing rate and the force, resulting in 5 values of the coefficient of determination ($R^{2}$ ). At last, the separation vector corresponding to the largest $R^{2}$ value was selected as the only separation vector applied to all forearm positions (termed the NRL-sinSV method). This procedure was repeated until the optimal separation vectors of all MUs from all fingers were found.

4) Multivariate Linear Regression Model

A multivariate linear regression model for individual fingers was built to map MU firing rates to finger forces:

$$\begin{equation*} Force^{j}=\sum \limits _{i} {a_{i} FR_{i}^{j} +b} \tag {3}\end{equation*}$$ with $j\in$ {index, middle, ring-pinky} and $FR_{i}^{j}$ as the firing rate of the $i$ th MU of finger $j$ , and only the single-finger trials were used to estimate the coefficient $a_{i}$ and $b$ for individual fingers independently. For a given finger $j$ , the EMG data from all the single-finger trials were collected and the corresponding spike trains were obtained using the NRL-mulSV and NRL-sinSV methods, respectively. In the NRL-mulSV method, different separation vectors were used for a MU based on the forearm position of the corresponding trial. In the NRL-sinSV method, the selected optimal separation vector was used consistently for an MU. The firing rate and the force were obtained using the sliding window and all the firing rate and force values from all forearm positions were used to perform the multivariate linear regression analysis using (3) for the NRL-sinSV method and the NRL-mulSV method, respectively.

5) Online Finger Force Prediction

The EMG signals of the multi-finger trails from all forearm positions were used to evaluate the force prediction performance. The EMG signals were segmented with the sliding window for online decomposition. Within each window, the EMG signals were extended and whitened, and the spike trains of individual MUs for individual fingers were then obtained via both the NRL-sinSV and NRL-mulSV methods. The obtained firing rate values of individual MUs were put into the multivariate linear models to predict the force of individual fingers. Finally, the predicted force was smoothed with a Kalman filter.

C. Force Prediction Using EMG Amplitude

EMG amplitude (root mean square, RMS) information was also used to predict multi-finger forces for comparison in this study (termed the EMG-Amp method). Same as the two neural drive-based methods, the single-finger trials were used to optimize the selection of EMG channels for individual fingers and train the multivariate linear regression model, and the multi-finger trials were used to validate the performance of force prediction. Figure 3 illustrates the EMG amplitude distribution during the extension of the index, middle and ring-pinky fingers, respectively across forearm positions from a representative subject. The heatmap shows that the activated regions overlapped a lot between fingers. Meanwhile, the activated region shifted when the forearm position changed. Therefore, a channel selection procedure was needed to select the optimal channels to calculate the EMG amplitude for individual fingers. The EMG amplitude values in the single-finger trials were obtained using the sliding window for individual channels and the original force signals of three fingers were also processed using the sliding window. The Pearson correlation coefficient $\bar {r}_{i,p}^{j,k}$ was calculated between the EMG amplitude of channel $i$ and the forces of finger $p$ using the single-finger trial of finger $j$ at forearm position $k$ . The EMG signals of channel $i$ would be used for finger $j$ if $\bar {r}_{i,j}^{j,k} \gt \bar {r}_{i,p_{1}}^{j,k}$ and $\bar {r}_{i,j}^{j,k} \gt \bar {r}_{i,p_{2}}^{j,k}$ with $j\ne p_{1} \ne p_{2}$ . This procedure was first performed for individual forearm positions separately, resulting in three channel groups for three fingers at each forearm position. Then, the intersection of the channel groups for the same finger across forearm positions was obtained as the optimal channels for individual fingers.

Fig. 3.

The normalized EMG amplitude distribution by the maximum for individual fingers across 5 forearm positions. The “$\times$ ” represent the centroids of the amplitude distribution.

Then a linear regression model for individual fingers was built to map the EMG amplitude to the finger force:

$$\begin{equation*}Force^{j}=\bar {a}Amp^{j}+\bar {b} \tag {4}\end{equation*}$$ with $j\in \{\textrm {index,}\,\textrm {middle,}\,\textrm {ring-pinky}\}$ and $Amp^{j}$ as the average EMG amplitude across all optimal channels for finger $j$ .

For the online force prediction phase, the EMG amplitude was calculated within the sliding window and put into the linear regression model to obtain the predicted force of each finger. Finally, the predicted force was smoothed with the same Kalman filter.

D. Evaluation of Force Prediction Performance

The force prediction performance was evaluated by comparing the predicted force using different methods with the recorded force data in the multi-finger trials. Two measurements were used including the root mean square error (RMSE) and the $R^{2}$ . In each multi-finger trial, the RMSE and the $R^{\mathrm {2}}$ value were calculated for individual fingers independently, resulting in three RMSE and three $R^{2}$ values. The two measurements were first compared using the repeated measures ANOVA to verify whether significant differences existed across the three methods and the post-hoc t-test with the Holm–Bonferroni correction was used to compare any pair of the three methods. The significant level was set at 0.05.

SECTION III. Result

A. Performance of Finger Force Prediction

Figure 4 illustrates the force prediction results from a representative multi-finger trial with little co-activation of fingers. The MU spike trains and the corresponding firing rates employed in force prediction of the index, middle, and ring-pinky fingers in the NRL-mulSV method and the NRL-sinSV method were presented in Fig. 4(a)-(c), respectively. The predicted finger forces using three methods and the corresponding recorded force was shown in Fig. 4(d) for individual fingers. The results showed that the EMG-Amp method cannot tell the extending finger, always leading to the overestimation of the other two fingers’ force when a specific finger exerted force. Compared with the NRL-mulSV method, the NRL-sinSV method had more underestimation issues, possibly because the inappropriate separation vectors resulted in the quality degradation of the source signal. Then, fewer spike events can be identified and the firing rate was underestimated.

Fig. 4.

The online force prediction results of a representative multi-finger trail at the 90° forearm position. (a)-(c) The MU spike trains and firing rates used to predict the force of index, middle, and ring-pinky fingers in the NRL-mulSV method and the NRL-sinSV method. (d) The comparison between the predicted force using the three methods and the recorded force.

1) Overall Performance

In order to evaluate the overall performance of the three methods, the $R^{2}$ and RMSE values were first averaged across fingers and then averaged across trials from all forearm positions for individual subjects as shown in Fig. 5(a) and 5(b). The repeated measures ANOVA indicated that the method had a significant influence on the $R^{2}$ (F(2,18) =13.79, p =0.0002) and RMSE (F (2,18) =30.05, p <0.0001) values. Further post-hoc t-test with the Holm–Bonferroni correction showed that the $R^{2}$ of the NRL-mulSV method was significantly larger than the EMG-Amp method (p <0.01) and the NRL-sinSV method (p <0.001). The RMSE of the NRL-mulSV method was significantly lower than the other two methods (p <0.001), and the RMSE of the NRL-sinSV method was significantly lower than the EMG-Amp method (p <0.01).

Fig. 5.

The average ${R} ^{{2}}$ (a) and RMSE (b) between the predicted and actual forces. The average ${R} ^{{2}}$ (c) and RMSE (d) between the predicted and actual forces for individual fingers. (e) The average ${R} ^{{2}}$ across forearm angles for the three methods. $^{\ast }$ , p<0.05, $^{\ast \ast }$ , p<0.01, $^{\ast \ast \ast }$ , p<0.001.

2) Performance for Individual Fingers

To further compare the force prediction performance of different methods for individual fingers, the $R^{2}$ (Fig. 5(c)) and RMSE (Fig. 5 (d)) values of individual fingers were averaged across all the multi-finger trials for individual subjects. The repeated measures ANOVA was performed and revealed that the method had a significant influence on the $R^{2}$ (F(2,54) =18.53, p <0.0001) and the RMSE (F(2,54) =16.16, p <0.0001). However, there was no significant interaction between the method and finger factors on the $R^{2}$ (F(4,54) =0.93, p >0.05) and the RMSE (F(4,54) =2.06, p >0.05). The post-hoc test with the Holm–Bonferroni correction showed that the $R^{2}$ of the NRL-mulSV method was significantly larger than the NRL-sinSV method (p <0.05) and the EMG-Amp method (p <0.01) for both the index and the middle fingers. For the ring-pinky finger, the $R^{2}$ of the NRL-mulSV method was significantly higher than the NRL-sinSV method (p <0.05) but showed no significant superiority compared with the EMG-Amp method (p =0.15). The RMSE of the NRL-mulSV method for the index was significantly smaller than the NRL-sinSV method (p <0.05) and the EMG-Amp method (p <0.01). The RMSE of the NRL-mulSV method for the middle was significantly smaller than the EMG-Amp method (p <0.05). As for the ring-pinky finger, there were no significant differences in the RMSE between any pair of the three methods (p >0.05).

In the multi-finger trials, different fingers extended one after the other. Although co-contractions were allowed, the co-contraction level was low for some subjects. In order to further verify the performance of the proposed method when strong muscle co-contractions exist, four more subjects were recruited and finished the multi-finger trials in which all the fingers need to extend simultaneously. The results showed that the NRL-mulSV can still obtain the best performance compared with the other two methods (Supplementary Materials).

3) Performance at Different Forearm Positions

In order to clarify if there were differences in the force prediction performance across forearm positions, the $R^{2}$ values in different forearm positions were averaged across trails from all fingers. The one-way ANOVA was performed on the $R^{2}$ values of the three methods separately. The results showed that the forearm position had a significant influence on the ${R} ^{2}$ values using the NRL-sinSV method (F(4,36) =3.22, p =0.024) and had no significant influence on the NRL-mulSV method (F(4,36) =0.59, p =0.67) and the EMG-Amp method (F(4,36) =0.97, p =0.43). The post-hoc test with the Holm–Bonferroni correction showed that, for the NRL-sinSV method, the $R^{2}$ at the 90° forearm position was significantly larger than that at the 30° (p <0.01), 60° (p <0.05), and 150° (p <0.05) forearm position.

B. Effects of Forearm Rotation

1) Centroid of EMG Amplitude Distribution

To examine the effect of forearm rotation on the EMG activities, the centroid shift of the EMG amplitude distribution between forearm positions was analyzed. The centroid of the EMG amplitude distribution was first estimated for individual forearm positions using the corresponding single-finger trials (Fig. 3). Then, the centroid shifts between any two forearm positions (e.g., 30° and 120°) were calculated and divided into different groups based on the corresponding angle displacement (e.g., $120^{\circ } - 30^{\circ }= 90^{\circ }$ ) between the two forearm positions. Next, the centroid shift values were averaged within the same group for individual fingers and individual subjects (Fig. 6(a)). As expected, the shift increased as the angle displacement between forearm positions got larger. The further two-way (finger vs. angle displacement) ANOVA demonstrated that the angle displacement factor had a significant influence on the centroid shift (F(3,27) =5.80, p =0.004)while the finger factor had no significant influence (F(2,18) =1.43, p >0.05). The interaction between the angle displacement and finger was not significant (F(6,54) =0.97, p >0.05).

Fig. 6.

The influence of the forearm rotation angle displacement on the centroid shift of the EMG amplitude distribution (a), the MUAP shape variation (b) and the online decomposition accuracy (c).

2) Variation of MUAP Profiles

The MUAP shape variation across forearm positions was also evaluated. The MUs that can be captured at all 5 forearm positions were selected for this analysis (96 MUs in total). The MUAP template was calculated using the spike-triggered averaging (STA) method [39] for individual forearm positions. Then, the correlation coefficient of MUAP waveforms was calculated for individual channels and the weighted average of the correlation coefficient across channels was calculated to quantify the similarity of MUAP profiles of the same MU between any two forearm positions. Lastly, the MUAP correlation coefficients with the same angle displacement were averaged across MUs for individual subjects. The results showed that as the forearm rotated over a larger angle displacement, the MUAP profile variation became larger, manifested as the decreased MUAP correlation coefficient (Fig. 6(b)). The two-way ANOVA (angle displacement vs. finger) demonstrated that the angle displacement factor had a significant influence on the MUAP correlation across forearm positions (F(3,27) =8.82, p <0.001) the finger factor had no significant influence (F(2,18) =0.25, p >0.05). And there was a significant interaction between the angle displacement and finger factors (F(6,54) =2.76, p =0.02).

3) Decomposition Accuracy

As mentioned in the introduction, the applicability of the separation vectors extracted at a specific forearm position when applied to other forearm positions is critical because we cannot extract the separation vectors corresponding to all successive forearm positions for a MU in realistic applications. Therefore, it is necessary to investigate the influence of the angle displacement between the forearm positions for separation vector extraction and online decomposition, respectively on the decomposition accuracy. The same 96 MUs were used for this analysis and their spike trains corresponding to individual separation vectors obtained during the offline decomposition phase were taken as the “ground-truth”. The spike trains were then obtained in the online decomposition manner by applying the separation vectors from a specific forearm position to the same forearm position or other forearm positions. The resultant spike trains were compared with the corresponding “ground-truth” and the spike consistency was estimated as the accuracy of online decomposition. Similarly, the calculated SC values were divided into different groups according to the angle displacement between the forearm position for separation vector extraction and the forearm position for online decomposition and then averaged across MUs for individual fingers and individual subjects (Fig. 6(c)). The results showed that the online decomposition accuracy decreased as the angle displacement increased. The two-way ANOVA (angle displacement vs. finger) showed that the angle displacement had a significant influence on the online decomposition accuracy (F(4,36) =31.33, p <0.0001) and the finger factor had no significant influence (F(2,18) =0.63, p >0.05). The interaction between the angle displacement and finger was not significant (F(8,72) =1.50, p >0.05). This demonstrated that for all three fingers, the online decomposition accuracy at a specific forearm position got improved if the separation vector obtained from a nearer forearm position can be used.

C. Applicability Validation of Separation Vector

In order to further verify the applicability of separation vectors extracted at some specific forearm positions when they are applied to other forearm positions, the three methods were compared via the following procedure. Specifically, the single-finger trials from the 30°, 90° and 150° forearm positions were used for offline initialization of the three methods, and the multi-finger trials from the 60° and 120° were used for online force prediction (Fig. 7(a)). The steps for the offline initialization and online force prediction were the same as mentioned in the Method section. The resultant $R^{2}$ (Fig. 7(b)) and RMSE (Fig. 7(c)) values of individual fingers were calculated and averaged across all the multi-finger trials for individual subjects. The repeated measures ANOVA was performed and revealed that the method had a significant influence on the $R^{2}$ (F(2,54) =14.37, p <0.0001) and the RMSE (F(2,54) =15.56, p <0.0001). The post-hoc test with the Holm–Bonferroni correction showed that the $R^{2}$ of the NRL-mulSV method was significantly larger than the NRL-sinSV method (p <0.05) and the EMG-Amp method (p <0.01) for both the index and the middle fingers. For the ring-pinky finger, there were no significant differences between any pair of the three methods (p >0.05). The RMSE of the NRL-mulSV method was significantly smaller than the other two methods (p <0.05) for the index and middle fingers. For the ring-pinky finger, The RMSE of the NRL-mulSV method was significantly smaller than the NRL-sinSV method (p <0.05) but showed no significant superiority compared with the EMG-Amp method (p >0.05).

Fig. 7.

The applicability of separation vectors is further verified by using the single-finger trials from the 30°, 90° and 150° forearm positions for offline initialization, and the multi-finger trials from the 60° and 120° forearm positions for online force prediction (a). The average ${R} ^{{2}}$ (b) and RMSE (c) between the predicted and actual forces for individual fingers. $^{\ast }$ , p<0.05, $^{\ast \ast }$ , p<0.01.

SECTION IV. Discussion

Although a number of previous studies have demonstrated that the motoneuron discharge information can obtain better performance on motion decoding compared with the conventional global EMG features, the factors that can lead to the non-stationary of the MUAP profiles could easily eliminate the superiority [30]. In a realistic application scenario, forearm rotation is one of the most prominent factors that can lead to the variation of the MUAP waveforms from the extensor/flexor digitorum muscle, resulting in the lower EMG decomposition accuracy and then the degradation of dexterous motion decoding. This study aimed to find an effective way to extract MU discharge information accurately across forearm positions for the online prediction of multi-finger dexterous forces. Basically, the proposed NRL-mulSV method supposed that, for a given forearm position, the MU features, i.e., separation vector, extracted from the nearer forearm position was typically more accurate than the one extracted from a farther forearm position. Therefore, multiple separation vectors were extracted at different forearm positions for each MU and the one extracted at the nearest forearm position was always used for online decomposition. As comparison, the NRL-sinSV method always used the single separation vector extracted at the ‘optimal’ forearm position. The overall results showed that the NRL-mulSV method showed significant superiorities compared with both the NRL-sinSV and the EMG-Amp methods, while the superiority of the NRL-sinSV method compared with the EMG-Amp method was not significant under some circumstances. These results indicated that the proposed method improved the robustness of the neural drive-based method for motion decoding and could potentially facilitate its application in the realistic NMI system.

The two neural drive-based methods, especially the NRL-mulSV method showed significantly improved performance compared with the EMG-Amp method in dexterous multi-finger force prediction. The EMG crosstalk, muscle coactivation, and limited sampling depth have been demonstrated to compromise the ability to estimate dexterous motor intention [5]. The forearm rotation that would vary the relative position between muscle fibers and electrodes can vary the EMG amplitude distribution (Fig. 6(a)) and further intensify the crosstalk issue [29]. Also, the degree of muscle coactivation and the musculoskeletal geometry varied across forearm postures [28], leading to more complicated components of crosstalk. In addition, a previous study found that the forces were reduced when the forearm deviated from the neutral position, but EMG amplitude did not always reflect the changes in force [29]. All these make it more challenging to select the appropriate EMG channels to predict the force of individual fingers independently. Our results showed that the EMG-Amp method failed to distinguish individual finger forces (Fig. 4(d)) even though an optimization procedure had been performed to select the EMG channels for individual fingers. As for the neural drive-based method, the MUs can be classified into individual fingers according to the correlation between MU firing rate and finger force proposed in the previous study [19], [38]. This procedure helped to eliminate the impact of the cross-talk issue because the EMG amplitude information was not used in the neural drive-based method (Fig. 4(d)).

Comparing the two neural drive-based methods, the NRL-mulSV method outperformed the NRL-sinSV method significantly. This indicated that the constant separation vector extracted at a specific forearm position, even after the optimization procedure, was still not accurate enough to decompose EMG signals from other forearm positions for accurate multi-finger force prediction. Our results showed that the MUAP profile variation became larger when the forearm rotated over a larger angular displacement (Fig. 6(b)). This further led to the consequence that the online decomposition accuracy decreased more if the separation vector extracted from a farther forearm position was used (Fig. 6(c)). For the NRL-sinSV method, the force prediction performance showed significant differences between forearm positions (Fig. 5(e)). Specifically, the $R^{\mathrm {2}}$ at the 90° forearm position was higher compared with other forearm positions, possibly because the 90° forearm position, i.e., the neutral position had the smallest average angle displacement to other forearm positions, resulting in the highest average decomposition accuracy. These results indicated the superiority and necessity to vary the separation vector according to the forearm positions.

Overall, the influence of the forearm rotation on the centroid shift of the EMG amplitude distribution, the MUAP shape variation and the online decomposition accuracy showed consistency across fingers (Fig. 6). Basically, as the rotation angle displacement increased, the centroid shift and the MUAP variation increased, and therefore the online decomposition accuracy decreased. When comparing the three fingers, the centroid shift and the MUAP variation of the middle fingers were relatively smaller compared with the other two fingers although not significant. This possibly explains why the middle finger can obtain relatively better force prediction performance compared with the other two fingers on average for three methods.

The forearm position changes continuously and we cannot extract the separation vector at every successive forearm position for a MU. Therefore, there should be a clear strategy about how to use the limited number of separation vectors for all possible forearm positions. Although some previous studies proposed to use separate motion decoding models for individual arm positions [31], [32], [33], the data for model training and performance testing are from the same arm positions, and therefore, the applicability of the decoding model to other arm positions is not known. In this study, it has been demonstrated that the online decomposition performance of the separation vectors would decrease when they were used at a farther forearm position. Therefore, a simple strategy could be obtained that the separation vector from the nearest forearm position should always be used, in order to obtain the best performance. The effectiveness of the strategy was further verified by comparing the three methods when the data for offline initialization and online force prediction were not the same (Fig. 7). Although the selection of the number (three) and combination (30°/90°/150°) of forearm positions used to extract the separation vectors in this study was, to some degree, arbitrary, the effectiveness of the proposed strategy should not be affected. The major reason was the performance of the separation vector relies on the angle displacement between the position where it was used and where it was extracted rather than the specific position where it was extracted only as demonstrated in the results (Fig. 6).

One limitation of this study was that only a limited number of forearm positions without complete pronation and supination were tested and the EMG and force signals were not recorded during continuous forearm rotation, mainly because there was a varying passive pressure onto the load cells from fingers, resulting in a continuous slow drift of the offset of force signals. However, our results showed that the online decomposition accuracy can be improved by using the separation vectors extracted from a nearer forearm position (Fig. 6(c)). This indicated that the strategy proposed in our study has the potential to deal with continuous forearm rotation only if separation vectors can be extracted at more forearm positions. In addition, the NRL-mulSV method relies on the knowledge of the forearm position, and the method to acquire the forearm position information in a realistic application was not involved in this study. It is planned to utilize the IMU sensors to measure the forearm rotation angles, which will be tested in our further study on the online decoding and control.

SECTION V. Conclusion

This study provided a solution to extract MU discharge information accurately across forearm positions for dexterous multi-finger force prediction in real-time. The results showed that the proposed NRL-mulSV method using the neural drive information extracted with multiple separation vectors obtained the best performance compared with the conventional EMG amplitude information and the neural drive information extracted with a single constant separation vector. Our results demonstrated the robustness of the proposed method for dexterous finger force prediction during forearm rotation, which could potentially promote the application of the EMG decomposition technique for continuous motion decoding in a realistic NMI application scenario.