Data-Based and Opportunistic Integral Concurrent Learning for Adaptive Trajectory Tracking During Switched FES-Induced Biceps Curls

Hybrid exoskeletons, which combine functional electrical stimulation (FES) with a motorized testbed, can potentially improve the rehabilitation of people with movement disorders. However, hybrid exoskeletons have inherently nonlinear and uncertain dynamics, including combinations of discrete modes that switch between different continuous dynamic subsystems, which complicate closed-loop control. A particular complication is the uncertain muscle control effectiveness associated with FES. In this work, adaptive integral concurrent learning (ICL) motor and FES controllers are developed for a hybrid biceps curl exoskeleton, which are designed to achieve opportunistic and data-based learning of the uncertain human and electromechanical testbed parameters. Global exponential trajectory tracking and parameter estimation errors are proven through a Lyapunov-based stability analysis. The motor effectiveness is assumed to be unknown, and, to help with fatigue reduction, FES is enabled to switch between multiple electrodes on the biceps brachii, further complicating the analysis. A consequence of switching between the different uncertain subsystems is that the parameters must be opportunistically learned for each subsystem (i.e. each electrode and the motor), while that subsystem is active. Experiments were performed to validate the developed ICL controllers on twelve healthy participants. The average (± standard deviation) position tracking errors across each participant were 1.44 ± 5.32 deg, −0.25 ± 2.85 deg, and −0.17 ± 2.66 deg across biceps Curls 1-3, 4-7, and 8-10, respectively, where the average across the entire experiment was 0.28 ± 3.53 deg.


I. INTRODUCTION
. 38 However, closed-loop FES control of muscle effort is chal-39 lenging since the muscle effectiveness is unknown, the muscle 40 dynamics are both nonlinear and uncertain, and high stimu-41 lation inputs are often uncomfortable [5], [6]. Furthermore, 42 rehabilitative hybrid exoskeletons, which combine FES and 43 motor control, must alternate control between FES and a motor 44 without compromising performance. 45 Closed-loop FES control has previously been implemented 46 on a range of rehabilitative exercises, such as rowing [7], 47 cycling [8], [9], [10], [11], [12], walking [13], leg exten-48 sions [14], [15], [16], [17], and biceps curls [1], [2], [3], 49 [4], among others. To compensate for system uncertainties, 50 and to ensure stability, many closed-loop FES controllers 51 have included only robust (i.e., high (infinite) frequency 52 and/or high-gain) feedback terms (cf. [1], [2], [3], [16], 53 [17]). An added motivation for such robust controllers is that 54 they often produce a negative definite derivative of a strict 55 Lyapunov function, which aids the stability analysis of a 56 switched system (i.e., a system with mixed continuous and 57 discrete dynamics, also called a hybrid system). However, the 58 high-gain/high-frequency nature of robust FES control tends 59 to increase the rate of fatigue and may also be uncomfortable 60 for the participant [ [14], [15]) to ensure asymptotic trajec-64 tory tracking. The adaptive controllers in [7], [8], [9], [10], 65 [11], and [14] implemented model-free techniques ranging 66 from repetitive (RLC) and iterative (ILC) learning control, 67 neural networks (NN), and fuzzy logic, whereas [12], [13], 68 [15], [18], [19] implemented model-based techniques. 69 Although adaptive controllers are often used to improve 70 control performance, sometimes it is desired for the adaptive 71 This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ controller to simultaneously ensure parameter/system identi-72 fication. Traditional adaptive controllers (cf. [20], [21]) can 73 yield both exponential tracking and parameter identifica-74 tion provided the persistence of excitation (PE) condition is ing input/output data for each subsystem, while that subsystem 126 is active. After sufficient learning has occurred for a given 127 subsystem, its parameters are able to be updated regardless of 128 the currently active subsystem. Comparative experiments were 129 performed on twelve healthy participants using the developed 130 control system, a traditional adaptive controller, and a robust 131 controller resulting in average (± standard deviation) position 132 tracking errors of 0.28 ± 3.53 deg, 1.47 ± 5.78 deg, and 133 3.36 ± 7.97 deg, respectively, across a 10 curl experiment. 134 The results indicated improved tracking performance for the 135 ICL controller compared to some traditional adaptive and 136 robust controllers, while providing similar average FES and 137 motor control inputs. The results further demonstrated the 138 ability of the ICL controller to improve the tracking perfor-139 mance as adaptation occurred. Efforts to perform preliminary 140 experiments on participants with neurological conditions were 141 stymied due to  II. DYNAMICS 143 The dynamics of the uncertain nonlinear hybrid biceps curl 144 exoskeleton are modeled as 1 where q : R ≥0 → Q,q : R ≥0 → R, andq : R ≥0 → R 147 denote the measured angle, measured angular velocity, and 148 unmeasurable acceleration, respectively, of the forearm about 149 the elbow joint. The set Q ⊂ R denotes a compact set of 150 potential forearm angles. The inertial, gravitational, passive 151 viscoelastic tissue, and damping effects of the hybrid biceps 152 curl exoskeleton are denoted by M : R → R >0 , G : Q → R, 153 P : Q × R → R, and B d : R → R, respectively, and are 154 defined as where J, m, g, l, b v , b d ∈ R >0 and k e1 , k e2 ∈ R are unknown 158 constants and θ 0 ∈ R >0 is a known constant.

159
In this paper, FES is applied via multiple electrodes that 160 are placed on the biceps brachii muscle using w ∈ N distinct 161 channels of a stimulator, where m ∈ M {1, 2, . . . , w} 162 indicates the m th electrode channel, and M is a finite set. 163 The torques produced about the elbow joint due to the motor 164 and FES-induced muscle contractions are denoted by τ e , τ M : 165 R ≥0 → R, respectively, and defined as where Q m ⊆ Q denotes the set of angles over which the which can be rewritten as

211
where Y 1 ∈ R 1× p denotes a known regression matrix, θ i ∈ 212 R p denotes the unknown constant parameters for the i th 213 subsystem, and p denotes the number of uncertain parameters.

214
3 FES-induced muscle activation can only produce a positive torque and negative FES inputs are set to zero during implementation. Therefore, the switching signals in (6) and (7) could result in uncontrolled regions; however, this situation was not observed during the subsequent experimental analysis. If desired, the switching signals in (6) and (7) could be modified to set σ m = 0 if u m ≤ 0 and to set σ e = 1 if σ m = 0, ∀m ∈ M. 4 For notational brevity, all functional dependencies are hereafter suppressed unless required for clarity of exposition. The objective of this paper is for the forearm to track 217 a desired position and velocity. The position tracking error, 218 e 1 : R ≥0 → R, is measurable and is defined as (11) 220 An auxiliary tracking error, e 2 : R ≥0 → R, is measurable and 221 is defined as where α ∈ R >0 is a selectable constant.
To obtain the open-224 loop error system for the i th subsystem (i.e., when σ i = 1), 225 we take the derivative of (12), multiply both sides by J , and 226 use (9) to yield The parameter identification error vector for the i th subsys-233 235 whereθ i ∈ R p denotes the parameter estimates for the i th 236 subsystem. Based on the subsequent stability analysis, an 237 update law for the i th subsystem's parameter estimates is 238 designed ∀i ∈ S as where i ∈ R p× p is a user-selectable diagonal and positive 241 definite matrix, γ i ∈ R >0 is a selectable constant, and S i 242 contains a history stack of previous ICL terms, and is defined 243 ∀i ∈ S as where N i ∈ N denotes the size of the history stack for the 246 i th subsystem. The switching signal, σ i,l : R ≥0 → {0, 1}, 247 is designed to indicate when sufficient learning has been 248 achieved for the i th subsystem, and is defined ∀i ∈ S as by substituting (10) into (9), integrating both sides, and 270 then using the definitions in (19) and (20). Now a 271 non-implementable form of S i can be obtained, to facilitate 272 the subsequent stability analysis, by substituting (21) into (17) 273 and using (15) to yield

275
Notice that the acceleration measurements are included 276 in (10). However, an advantage of ICL compared to CL is 277 that the ICL terms (19)- (20) are designed in such a way 278 that acceleration is not required. The term (19) is obtained for the i th subsystem by integrating 280 both sides of (10) to yield 291 ∀i ∈ S, where k i ∈ R >0 , ∀i ∈ S are selectable constants. The 292 closed-loop error system for the i th subsystem is obtained by 293 substituting (26) into (13) to yield A special characteristic of the update laws for the parameter 298 estimates of each subsystem, as defined in (16), is that the 299 typical PE criteria can be relaxed to yield a FE criteria 300 for parameter estimation convergence, which is stated in 301 Assumption 1.

302
Assumption 1: Sufficient excitation for the i th subsystem 303 occurs over a finite duration of time. Thus, ∃T i ∈ R >0 , ∀i ∈ S 304 such that ∀t ≥ T i learning is complete for the i th subsystem 305 (i.e., σ i,l = 1), or in other words the following FE condition 306 is satisfied: λ min To facilitate the subsequent analysis, we define a common 310 Lyapunov function candidate, V : R 2+ p(w+1) → R ≥0 , that is 311 both continuously differentiable and positive define as Notice that (28) can be bounded as where λ, λ ∈ R >0 are known constants defined as Theorem 1: For the dynamic system in (9) with 321 Properties 1-3, the controllers defined in (26) and the 322 adaptive update laws defined in (16) ensure global bounded 323 parameter estimation and trajectory tracking errors for 324 t ∈ [0, T ), provided the following sufficient conditions are 325 met 6 Proof: Since the update laws in (16) and the closed-328 loop error system in (27) are discontinuous, the solution to 329 the time derivative of (28) exists almost everywhere (a.e.) 330 within t ∈ [t 0 , ∞). There exists a generalized time derivative 331 is defined as in [30], and h 334 ė 1ė2θ1θ2 . . .θ wθe T [31]. Taking the time derivative of 335 (28) and substituting in (12) yields

338
Consider the case when σ i = 1 for some i ∈ S such that 339θ i , ∀i ∈ S and Jė 2 are continuous according to (16) and (27).

345
for the case when σ i = 1. When σ i = 1, (16) and (22) can 346 be used to determine that for a given k ∈ a.e.

361
By inspection of (28) provided the conditions in (31) are satisfied.

378
Proof: First, consider the time interval t ∈ [T, ∞). Notice 379 that λ min , σ i,l = 1, ∀i ∈ 380 S) by Assumption 1, and hence, . For the 382 case when σ i = 1, for some i ∈ S, (33) can be rewritten by 383 using (16) and the fact that   The adaptive update law in (16) contains both ICL terms 407 (γ i i S i ) and more traditional adaptive terms ( i Y T 2 e 2 ). Note 408 that the ICL terms could be removed by setting γ i = 0, ∀i and 409 all adaptive terms could be removed by setting i as a matrix 410 of zeros ∀i . Hereafter, the developed control system in this 411 work (i.e., (16) and (26)) is referred to as Controller A, the 412 developed adaptive controller without ICL terms (γ i = 0, ∀i ) 413 is referred to as Controller B (i.e., a traditional adaptive 414

452
During each experiment, the arm was initially fully extended 453 (i.e., q(t 0 ) = 0 deg) and the desired angular position was 454 defined as The motor was used during the first 5 s to move forearm 457 to 25 deg, after which the next 125.6 s consisted of either 458 Controller A, B, or C being implemented to perform a total 459 of 10 arm curls between 25 deg and 95 deg.

460
Experiments were performed using each participant's dom-461 inant arm, and Controllers A, B, and C were implemented in 462 a random order. Participants were blind to the tracking per-463 formance during each experiment, and were asked to remain 464 passive and provide no volitional effort. For each participant, 465 a single experiment was performed using each controller.

466
As stated in Section III.B, data was recorded during the 467 experiments to calculate (19) and (20) for each subsystem. 468 Furthermore, to facilitate implementation a counter was devel-469 oped and initialized at zero for each subsystem. For the i th 470 subsystem, whenever both (19) and (20) were non-zero, the 471 counter for the i th subsystem was increased by one and then 472 the recorded values for (19) and (20) were included in the i th 473 subsystem's history stack in (17), until the history stack was 474 full (i.e., the counter was at N i ). At this point, the counter 475 was reset to zero. Subsequently, whenever both (19) and (20) 476 were non-zero, they were added to the history stack if they 477 increased the eigenvalue of the subsystem, otherwise the data 478 was discarded. During the experiments, the following history 479 stack parameters were implemented: λ i = 5 × 10 −6 , ∀i ∈ S, 480 N i = 1000, ∀i ∈ S, and t = 0.15 s. 481

482
Descriptive statistics of the position tracking error, motor 483 effort, and FES effort are included in Table I. To demonstrate 484 the effect of adaptation and to compare each controller, the 485 results in Table I are averaged across each participant for Curls 486 1-3, 4-7, 8-10, and 1-10 (i.e., the overall results). Across each 487 participant, the average (± standard deviation) position track-488 ing errors were 1.44 ± 5.32 deg, 2.84 ± 7.40 deg, and 3.79 ± 489 8.14 deg across Curls 1-3 for Controllers A, B, and C, 490 respectively, −0. 25   the parameter estimates in Fig. 4 include the estimates for  Table I:

534
In the second set of tests, Friedman tests were con-535 ducted to determine, for each controller, if the curl groups 536 (i.e., Curls 1-3, 4-7, and 8-10) affected each measure-537 ment and determined that the curl group had a sig-538 nificant effect on the median RMS position errors for 539 Controllers A (P-Value < 0.001) and B (P-Value = 540 0.001), the median peak position errors for Controllers A 541 (P-Value = 0.006) and B (P-Value = 0.039), and the median 542 SD of the FES effort for Controller A (P-Value < 0.001). 543 Two-sided paired Wilcoxon signed-rank tests with Bonferroni 544 corrections were performed on the significant measurements 545 from the second set of Friedman tests and it was concluded that 546 there was no significant difference between Curl group 4-7, 547 compared to Curl Using the data in Table I for Curls 1-10, Controller A, 567 compared to Controller B (Controller C), decreased 8 the RMS 568 position error by 42.4% (59.6%), the peak position error by 569 26.8% (34.3%), the mean motor effort by -1.3% (0.5%), the 570 SD of the motor effort by 4.2% (8.6%), the mean FES effort by 571 4.5% (4.3%), and the SD of the FES effort by 22.8% (24.2%). 572 These results can be visually observed in Figs. 2 and 3 for 573 a single participant. Furthermore, the statistical analysis con-574 firmed that Controller A reduced the median position tracking 575 error and the median SD of the FES effort relative to Con-576 trollers B and C and that Controller B improved the position 577 tracking performance relative to Controller C. Therefore, it is 578 clear that the adaptive controllers (Controllers A and B) 579 outperformed a robust controller (Controller C) in position 580 tracking; however, the addition of adaptive ICL terms (Con-    inputs tend to be larger for the former group [32], [33], [34]. 592 Therefore, it is expected that Controller A would outperform 593 Controllers B and C for participants with neurological con-594 ditions. Furthermore, the system identification performance is 595 unable to be evaluated because the actual system parameters 596 are unknown. However, visual inspection of Fig. 4 indicates 597 that different parameters were learned for Electrodes 1-3, 598 which was expected due to each electrode likely having a 599 different control effectiveness.

600
The effect of adaptation on position tracking can be investi-601 gated by comparing the results for each curl group in Table I. 602 In fact, from Curls 1-3 to Curls 4-7, the RMS position error 603 decreased by 46.8%, 29.0%, and 1.2% for Controllers A, B, 604 and C, respectively, and the peak position error decreased by 605 40.8%, 33.4%, and 3.6% for Controllers A, B, and C, respec-606 tively. In fact, the statistical analysis confirmed that the median 607 RMS and peak position errors decreased from Curls 1-3 to 608 4-7 and from Curls 1-3 to 8-10 for both Controllers A and 609 B. Furthermore, from inspection of Table I the RMS and 610 peak position errors changed minimally (|percent change| < 611 10%) from Curls 4-7 to 8-10 for each controller, which was 612   Table I. Overall, the motor effort had minimal changes 633 across each curl group for each controller, which is confirmed Controller A, the SD of the FES effort decreased by 16.2% 640 from Curls 1-3 to 4-7 and by 13.5% from Curls 4-7 to 641 Curls 8-10, and the statistical analysis confirmed that the FES 642 variance significantly decreased from Curls 1-3 to Curls 4-7 643 and from Curls 1-3 to Curls 8-10. Another important obser-644 vation is that the position tracking improved significantly 645 between Curls 1-3 and Curls 4-7 for Controller A, but the FES 646 variation decreased and the median motor and FES efforts had 647 negligible changes from Curls 1-3 and Curls 4-7. Therefore, 648 the ICL-based adaptation was able to improve the tracking 649 performance and decrease the FES variance without increasing 650 the median control effort.

VIII. CONCLUSION 652
Adaptive ICL motor and FES controllers that use data-based 653 and opportunistic learning were developed for a hybrid 654 biceps curl exoskeleton. Global exponential trajectory track-655 ing and parameter identification were guaranteed through 656 a Lyapunov-like switched systems stability analysis. FES 657 was allowed to switch between multiple electrodes on the 658 biceps brachii and the motor effectiveness was uncertain, 659 which required a unique set of parameters to be oppor-660 tunistically learned for each subsystem. Experiments were 661 performed on twelve healthy participants to compare the 662 developed control system, a traditional adaptive controller, 663 and a robust controller, which resulted in average position 664 tracking errors of 0.28 ± 3.53 deg, 1.47 ± 5.78 deg, and 665 3.36 ± 7.97 deg, respectively, across a 10 curl experiment.

666
A clinically significant feature of ICL is that the uncertain 667 human and testbed parameters can potentially be identified in