Co-adaptive control of bionic limbs via unsupervised adaptation of muscle synergies

— Objective: In this work, we present a myoelectric interface that extracts natural motor synergies from multi-muscle signals and adapts in real-time with new user inputs. With this unsupervised adaptive myocontrol (UAM) system, optimal synergies for control are continuously co-adapted with changes in user motor control, or as a function of perturbed conditions via online non-negative matrix factorization guided by physiologically informed sparseness constraints in lieu of explicit data labelling. Meth-ods: UAM was tested in a set of virtual target reaching tasks completed by able-bodied and amputee subjects. Tests were conducted under normative and electrode perturbed conditions to gauge control robustness with comparisons to non-adaptive and supervised adaptive myocon-trol schemes. Furthermore, UAM was used to interface an amputee with a multi-functional powered hand prosthesis during standardized Clothespin Relocation Tests, also conducted in normative and perturbed conditions. Results: In virtual tests, UAM effectively mitigated performance degradation caused by electrode displacement, affording greater resilience over an existing supervised adaptive system for amputee subjects. Induced electrode shifts also had negligible effect on the real world control performance of UAM with consistent completion times (23.91 ± 1.33 s) achieved across Clothespin Relocation Tests in the normative and electrode perturbed conditions. Conclusion: UAM affords comparable robustness improvements to existing supervised adaptive myocontrol interfaces whilst providing additional practical advantages for clinical deployment. Signiﬁ-cance: The proposed system uniquely incorporates neuro-muscular control principles with unsupervised online learning methods and presents a working example of a freely co-adaptive bionic interface.


Co-Adaptive Control of Bionic Limbs via Unsupervised Adaptation of Muscle Synergies
Dennis Yeung , Irene Mendez Guerra , Ian Barner-Rasmussen , Emilia Siponen , Dario Farina , Fellow, IEEE, and Ivan Vujaklija , Member, IEEE Abstract-Objective: In this work, we present a myoelectric interface that extracts natural motor synergies from multi-muscle signals and adapts in real-time with new user inputs.With this unsupervised adaptive myocontrol (UAM) system, optimal synergies for control are continuously coadapted with changes in user motor control, or as a function of perturbed conditions via online non-negative matrix factorization guided by physiologically informed sparseness constraints in lieu of explicit data labelling.Methods: UAM was tested in a set of virtual target reaching tasks completed by able-bodied and amputee subjects.Tests were conducted under normative and electrode perturbed conditions to gauge control robustness with comparisons to non-adaptive and supervised adaptive myocontrol schemes.Furthermore, UAM was used to interface an amputee with a multi-functional powered hand prosthesis during standardized Clothespin Relocation Tests, also conducted in normative and perturbed conditions.Results: In virtual tests, UAM effectively mitigated performance degradation caused by electrode displacement, affording greater resilience over an existing supervised adaptive system for amputee subjects.Induced electrode shifts also had negligible effect on the real world control performance of UAM with consistent completion times (23.91±1.33 s) achieved across Clothespin Relocation Tests in the normative and electrode perturbed conditions.Conclusion: UAM affords comparable robustness improvements to existing supervised adaptive myocontrol interfaces whilst providing additional practical advantages for clinical deployment.Significance: The proposed system uniquely incorporates neuromuscular control principles with unsupervised online learning methods and presents a working example of a freely co-adaptive bionic interface.

I. INTRODUCTION
U PPER limb loss caused by an accident, underlying disorder or a genetic condition can result in a reduction of function that impacts almost all aspects of daily living.In severe cases where clinical reconstructive techniques cannot provide sufficient functional improvement, technology-driven alternatives may be considered.With bionic restoration of human motor function, patients are fitted with robotic prostheses designed to mimic the capabilities of a missing extremity.In order to ensure the successful embodiment of these devices, the user's motor intentions must be reliably and faithfully interpreted [1].
Surface electromyography (EMG) from residual muscles is usually employed for establishing this human-machine interfacing [2], [3].The most basic and common form of myocontrol being the direct coupling between a single prosthetic function, typically hand open and close, and two recording sites.Whilst robust and effective in simple control tasks, conventional direct control no longer facilitates an interface that can efficiently engage the multiple grip patterns and wrist articulations that modern hand prostheses offer.However, by employing more sophisticated machine learning-based methods such as multichannel pattern recognition, users can directly access different degrees-of-freedom (DoFs) of their device [4].
Meanwhile, regression-based techniques establish a continuous mapping between EMG signals and prosthesis commands [5]- [8].These interfaces are essentially an extension of neuromotor control, involving model-specific transformations from muscle activation patterns to controller output [9].Indeed, the modular recruitment of muscle synergies observed in conventional motor tasks is also present during myoelectric control [10].Here, muscle synergies pertain to locally time-invariant weights associated to groups of muscles whose coordinated activity is driven by time-varying motor command primitives.Evidence exists that sparsity is an inherent attribute of the synergy framework, in both synergy composition and coordination [11], [12].Indeed, the latter property has been leveraged in establishing physiologically-inspired myocontrol [13].In addition to providing multi-functional control, these approaches promote motor learning by users since the enabled mapping is continuous and associated to natural motor modules [14]- [16].
Despite the possibility for the user to adapt to non-stationary signal characteristics [17], [18], some changes in signal features require an adaptation by the control algorithm.For example, the electrode displacement that occurs when redonning a prosthesis corresponds to a sudden change in signal properties that cannot be compensated by the user [17], [19].Multiple adaptive algorithms have thus been proposed to address the issue of control degradation where the system itself may be updated with new data if it becomes defective, as opposed to a full recalibration of the interface which is more disruptive and time-consuming.Here, incremental learning techniques are employed which can be categorized as either supervised (requiring data labels) or unsupervised (label-free adaptation).Many of these operate with designated offline data collection phases for model adaptation, where the resultant systems often exhibit higher robustness when compared to their non-adaptive counterparts [20]- [23].Furthermore, system adaptation can be implemented as an online procedure with the machine and user concurrently adapting in a cooperative manner during virtual target reaching tasks [24]- [26].These co-adaptive schemes can remedy defective function mappings by utilizing novel input data generated during online control.They leverage this new data to refine function spaces where no training samples were provided during system initialization.Currently, only one commercial myoelectric control system supports machine adaptation, and this is implemented as a user-initiated offline data gathering sequence [27].
Several drawbacks remain with current adaptive myocontrol solutions.First, offline data gathering interrupts continuous use and requires focused effort from the user throughout the procedure [20], thereby diminishing the convenience of these devices.Second, offline methods provide no feedback regarding model changes until the end of the procedure and the user is uninformed as to whether the new data they provide is beneficial.Third, adaptation schemes based on supervised learning require meticulous data labels.This can lead to complications associated with data mislabeling which may occur with users who cannot reproduce consistent muscle activation patterns to match provided cues [13].Lastly, in the case of online supervised co-adaptive methods, specialized equipment is required [24], making them more cumbersome to incorporate in commercial packages.
To address the aforementioned limitations, we propose an unsupervised adaptive myocontrol system with an online learning algorithm that is automatically administered in real-time.This approach forgoes disruptive re-training protocols and allows users to respond instantly to the evolving forward dynamics of the interface during device operation.Here, system adaptation is driven by an online extension of the non-negative matrix factorization (NMF) algorithm [28] which blindly identifies and refines basis functions analogous to muscle synergies.Thus, complications regarding data label obtainment and mislabeling are circumvented.While current unsupervised solutions operate on confidence-based acceptance of new data [20], [29], [30], our proposed solution leverages the physiological parallels of the control model and applies a-priori knowledge regarding the sparsity of motor command primitives [13] to guide the evolution of the interface, whereas the detection of conflicting command primitives, which may indicate a defective model, is used to trigger system adaptation.
We hypothesized that the proposed co-adaptive scheme would improve myocontrol performance and robustness by learning from the novel activation patterns produced by users during online operation.Furthermore, the system should exhibit capabilities of adapting its internal representations of motor synergies such that effects of signal non-stationarities can be alleviated.To test this, a series of virtual target reaching tasks were conducted with both able-bodied and amputee participants.The tests were done under the influence of different setup perturbations, where the proposed unsupervised adaptive myocontrol (UAM) scheme was compared with its non-adaptive counterpart (NAM) and a supervised adaptive myocontrol (SAM) scheme.Subsequently, to demonstrate the feasibility of employing UAM in real-world settings, the algorithm was integrated into the control of a commercial robotic hand prosthesis and operated by an amputee while performing standardized clinical tests.In both evaluation setups, the systems were tested under different conditions of electrode displacement to gauge their effectiveness in compensating for signal perturbations associated with the redonning of myoelectric interfaces.

A. NMF for Simultaneous and Proportional Myocontrol
Both UAM and NAM rely on an underlying myocontrol model based on NMF in which a series of muscle activation patterns, X, can be considered as an instantaneous linear mixture of command primitives, F, and their basis functions, W, which are analogous to muscle synergies [5], [13].
where n is the number of channels, k is the number of samples recorded for model initialization and m is the number of DoFs supported by the controller.Here, each DoF corresponds to a pair of motor tasks (i.e.flexion/extension) and is encoded by two opposing command primitives to satisfy the non-negativity constraint of NMF, hence . Given a set of activation patterns pertaining to a set of constrained motions, the latent variables W and F are extracted by iterating a pair of multiplicative update rules in equations ( 3) and (4), which optimize the cost function (2) in a block coordinate descent manner [31].
The procedure for controller initialization thus follows that in [5] and [32] where multi-DoF control models are constructed from combining independently extracted single-DoF synergy pairs.During online control, the estimated command primitives f (t) are obtained by iterating (5) with the current activation pattern x(t).The estimation is then scaled with subject-specific gains [5], [32] and smoothened with a seven point moving average filter [25] to give the controller output f (t).

B. Unsupervised Online Adaptation
To effectively update model basis vectors from user input in real-time, an online extension of the NMF algorithm must be employed as simply appending new data to the existing set of training samples and resolving the batch algorithm would lead to increasing computation times.Therefore, UAM is driven by an incremental approach to NMF which assumes that the optimality of command primitives factorized from past data is approximately unchanged when incremental changes are applied to model bases with each block of new data [28].Equation (6) shows this approximation in the cost function accounting for the first k samples from data of length k + l, where l is the size of the new samples.
Here, W k+l and F k+l represent optimal matrices from the factorization of X k+l while F k is the previously optimal encoding factorized from the first k columns of X k+l .As the cost function is separable by columns, its expression for the full set of input data, with the inclusion of the aforementioned approximation, can now be written as (7).
The introduced approximation is appropriate if k l such that the effect of new samples at each increment of adaptation is small.Hence, upon the arrival of a new block of samples, X l , the first k columns of F k+l are approximately equal to F k and only the last l columns, F l , need to be updated in conjunction with obtaining the new basis matrix W k+l .Under normative conditions of the synergy-inspired position control paradigm, the command primitives decoded from muscle activations should have some degree of sparseness.That is, each column of F l should mainly consist of near-zero elements with only a few strongly non-zero elements.While the L 0 norm directly relates to sparseness, its penalization is inconvenient to implement and, typically, the L 1 norm has been used to achieve a similar effect [13].Here, L 1/2 regularization was chosen as it has been shown to produce sparser solutions than L 1 regularization and can be efficiently incorporated to the multiplicative update scheme [19], [33].Adding this constraint to (7) results in (8).
where λ > 0 is the sparsity-promoting parameter imposed on the new block of command primitives.
Based on ( 8), the influence of new samples on model adaptation would diminish as data accumulates.This characteristic is undesired as the adaptation would become less effective over time.Therefore, the updating scheme requires a scaling mechanism such that older samples are gradually discounted as new data becomes available.
Here, 0 ≤ α o , α f ≤ 1 weigh the importance of old and new samples respectively such that successive applications of the adaptation algorithm R times will result in an exponentially decaying contribution from older sample blocks.This allows new synergies to be gradually learned and past works on co-adaptive simultaneous and proportional myocontrol have implemented similar concepts of 'forgetting' older data [24]- [26].
From the cost function in (9), the update rules for W k+l and F l may be derived under a gradient descent framework to yield (11) and ( 12) (detailed derivations in Appendix A). where denotes the element-wise inverse square root matrix of F l , that is, (F It can be seen in ( 12) that past information is retained in X k F k and F k F k .These storage matrices allow the space complexity of the adaptive algorithm to remain constant with regard to sample length and are updated through (13) and (14).

1) Adaptation Trigger:
The adaptation algorithm is designed to automatically initiate whenever defective control is detected.Here, the self-diagnosed trigger for adaptation is based on the rationale that position control schemes should not incur significant co-activation of opposing within-DoF commands.Hence, a set of thresholds, (δ + m , δ − m ), are set at 5% of the max primitive activation factorized from training data.The adaptation is thus triggered if both positive and negative thresholds of a DoF are exceeded at the same time instant.The pseudocode for system adaptation is shown in Algorithm 1 while Fig. 1 illustrates how UAM is integrated in a virtual test environment.
2) Parameter Selection: The proposed algorithm for UAM relies on four selectable parameters: regularization weighting λ, new sample block size l and sample weights α o and α f .An over-scaled sparsity constraint on the command primitives would produces excessive overlap between basis vectors while an under-scaled sparsity constraint would yield denser primitives with bases that are too sparse.In addition, a high α o value relative to α f would yield an unresponsive and ineffective adaptation while the opposite case would induce unstable, noise-prone adaptation [24].Following [28], α f was set to be (1 − α o ).Extensive pilot testing was then conducted to obtain suitable settings of λ = 5 and α o = 0.9 used for the experiments in this work.
For the parameter of l, larger values correspond to a larger repertoire of contiguous muscle activation patterns presented to the algorithm from which the model is updated with.The larger the set of activation patterns, the more accurately the model can update its new basis functions.Furthermore, a larger l increases the potential overlap of samples between consecutive blocks of adaptation input.Such overlap improves the stability of adaptation as the gradients of consecutive updates will be consistent.As such, it is favorable to incorporate more new samples than less, creating a trade-off between computational efficiency and the stability of learning.Here, l = 1000 was chosen which corresponds to activation data from the last 40 s since adaptation was triggered which allows for real-time adaptation that does not incur additional control latencies.

C. Experiments
1) Subjects: 10 able-bodied subjects, 7 male and 3 female, aged 22-34, all right-handed, with no prior experience in myocontrol participated in the virtual target reaching experiment.Two male amputee subjects, AM1 and AM2, aged 30 and 50, respectively, with below-elbow congenital deficiency on the Algorithm 1: Online NMF for UAM.
Initialize random non-negative matrix, F l 3: Initialize W k+l as W k 4: foriteration=1:iterationLimitdo 5: Update F l with (11) 6: Update W k+l with (12) 7: end for 8: Calculate X k+l F k+l with (13) 9: Calculate F k+l F k+l with (14) 10: end if 11: return W k+l left side also participated in the virtual experiment.Both use conventionally controlled myoelectric devices in their daily lives with approximately 24 and 8 years of experience for AM1 and AM2, respectively.Both amputee subjects have only had minimal exposure to advanced myocontrol prior to this study.In addition to the virtual experiment, AM2 also participated in the standardized Clothespin Relocation Tests.
All subjects provided their written informed consent.Both experiments were conducted in accordance with the Declaration of Helsinki and were approved by the research ethics committees of Aalto University and Imperial College London.
2) Data Acquisition and Processing: The Myo Armband (Thalmic Labs, Canada) was used to acquire EMG signals at a sampling frequency of 200 Hz.The root-mean-square (RMS) feature was extracted from the raw signal in windows of 160 ms length in time steps of 40 ms (120 ms overlap between consecutive windows).A custom Matlab (MathWorks Inc, MA, USA) framework was used to facilitate real-time processing of EMG, the implementation of myocontrol algorithms and online adaptation.
3) Model Training and Calibration: Prior to myocontrol assessment, training data was recorded for control model initialization.Subjects were seated in front of a computer screen with their arms relaxed by their sides.EMG was measured from the able-bodied subjects' dominant-side forearm while for amputee subjects, the signals were recorded from the affected limb.Three repetitions of each required single-DoF activation were recorded.For UAM and NAM, which share the same base control model, the procedure for initialization followed that of [5] while the model initialization approach used for SAM is described in [25].Here, activation patterns pertaining to wrist flexion and extension were assigned to DoF1.For all subjects but AM2, activations pertaining to radial and ulnar deviation were mapped to DoF2.For AM2, inability to reliably reproduce the default motions meant that activations for pronation and supination were used for DoF2 instead.After initialization, directional gains for boosting DoF activations were then manually tuned to ensure full accessibility of the usable solution space.
4) Virtual Target Reaching Tasks: In the virtual experiment, subjects were required to perform sets of target reaching tasks in a 2D space using different controllers and under different conditions of controlled electrode perturbation.Such virtual testing is well established in evaluating advanced myocontrol [5]- [7], [34].Here, each trial consisted of 24 predefined targets presented in randomized sequences with subjects given 10 s to reach each target, the spatial distribution of which ensured a thorough evaluation of multi-DoF simultaneous and proportional control performance.A more detailed description of the task parameters can be found in [19] while the task space layout is shown in Fig. 2.
NAM was assessed under normative trial settings while for the adaptive interfaces (UAM and SAM) an adaptation trial was first conducted to allow mutual co-adaptation of user and interface during task performance before a post-adaptation trial was conducted to assess the resultant control quality.During adaptation trials the online learning algorithm of the tested myocontrol scheme was enabled such that system adaptation could occur as the target acquisition tasks were being performed.For UAM, system adaptation was triggered whenever excessive conflicting commands were detected as described in Section II-B1, suggesting a function space that was poorly mapped.As this method is fully automatic, system adaptation was free to occur at  any point during the adaptation trial.For SAM, the myocontrol interface was deemed inadequate if a target could not be reached after 5 s at which point the supervised adaptation algorithm was triggered as implemented in [25].
To test the robustness of control to electrode shifting, trials were also repeated with electrodes transversally shifted from their original positions.The full trial sequence is shown in Table I.In medial-shifted trials, the sensor band was rotated such that the reference sensor pod was shifted closer to the medial epicondyle by approximately 1.2 cm while lateral-shifted trials were conducted with the sensor band displaced by the same magnitude in the opposite transversal direction (Fig. 3(a)).This displacement equates to half of the reference sensor pod width and is in alignment with past works where shifts from 0.8 cm up to 2 cm [17], [22], [23], [35] have been tested.

5) Prosthetic Hand Control and Clothespin Relocation
Test : The UAM algorithm was also incorporated as a controller for a commercial hand prosthesis.The Michelangelo Hand (Ottobock GmbH, Austria) with a rotatable wrist was used.A customized socket was built for the amputee AM2 with internal recesses to accommodate the wireless sensor band (Fig. 3(b)).The processing for UAM was done by the same Matlab framework with an additional module for the transmission of UAM's output to device servo commands via a Bluetooth link.This was done through a (proportional-integral-derivative) PID controller which minimized the difference between UAM's estimated DoF activation and the hand's actual pose as sensed by its internal encoders.Here, DoF1 was mapped to the opening and closing of the hand in a pinch grip pattern, while DoF2 was mapped to the rotation of the wrist.
While wearing the hand prosthesis, AM2 conducted the Clothespin Relocation Tests where three clothespins, located at head, chest and waist level, were moved from a vertical bar to a horizontal one.Here, to simulate the daily effects of electrode displacement in a controlled manner, control models were initialized with the sensor band rotated from its original position before donning the prosthesis.This method of simulating electrode displacement was required as, while this highly customized socket was worn, a rotation of the sensor band by 1.2 cm could not be reliably, nor comfortably maintained.
6) Performance Metrics: Control performance in the virtual experiment was gauged using four metrics common to Fitt's law style assessments [5], [7], [25].Completion rate (CR) measures the ratio of targets reached within the allotted time limit.Completion time (CT) measures the average time taken to reach a target during the trial.Path efficiency (PE) measures the ratio between cursor trajectory and target distance.Throughput (TP) measures the information transfer rate of the interface as a ratio between task difficulty and execution time.

D. Statistical Analysis
Two-factor repeated measures analysis of variance (RM-ANOVA) was performed on the metrics obtained from ablebodied subjects to identify any significant interactions between the factors of algorithm and electrode shift.Prior to this, normality of the collected data was verified with Shapiro-Wilk's tests and sphericity assumptions verified using Mauchly's test.If significant interaction between factors was detected from two-way RM-ANOVA, focused one-way RM-ANOVA was conducted for each level of both factors.If no significant interaction was detected from two-way RM-ANOVA, only the main effects were analyzed.In each of these tests, if the non-fixed factor was found to have significant effects, multiple pairwise comparisons were carried out with Bonferroni adjustment to identify statistically significant differences.

1) Able-Bodied Subjects:
The results from the virtual target reaching experiments are shown in Fig. 4. In all metrics, able-bodied subjects achieved better performance over NAM with both the supervised and unsupervised co-adaptive schemes.In normative conditions (Unshifted) marginally higher averages in CR, PE and TP and lower CT were observed in the postadaptation trials (post-UAM and post-SAM) with the UAM and SAM controllers.
When electrodes were transversally shifted by 1.2 cm both adaptive algorithms were effective in mitigating performance Fig. 5. Analysis of metrics with significant interaction between electrode condition and myocontrol scheme.Focused one-way RM-ANOVA was performed for each factor level of electrode condition and myocontrol scheme.Significant differences between factor levels are indicated by shaded blocks.
degradation.Significant interaction between the factors of algorithm and electrode shift was found (F 6,54 = 3.54, P = 0.005) in the CR results of able-bodied subjects.Focused one-factor RM-ANOVA revealed significant differences in CR values at different electrode perturbation states for NAM (F 2,18 = 9.83, P <0.001) with post-hoc analysis showing lower CR in both the lateral-shifted (−0.18,P = 0.012) and medial-shifted trials (−0.19,P = 0.003).Such results reinforce the notion that user adaptation alone is inadequate in mitigating the adverse effects of electrode displacement.Conversely, no significant drop in performance was detected for post-UAM nor post-SAM trials.Although a significant difference in CR was also detected for adapting UAM trials (F 2,18 = 7.264, P = 0.005), the magnitudes of decrease were lower (−0.13,P = 0.039 and −0.16, P = 0.017 for lateral and medial-shifted trials respectively) which indicate the UAM algorithm compensates for electrode shift in a gradual manner.When fixed on the lateral-shifted level, significant differences in CR were detected between the algorithms (F 3,27 = 13.099,P <0.001).Both post-UAM and post-SAM trials had higher CR (+0.17,P = 0.017 and +0.22,P = 0.010, respectively) than NAM.A significant difference between algorithms was also found at the medial-shifted level (F 3,27 = 7.184, P = 0.001), with higher CR from post-UAM and post-SAM trials (+0.13,P = 0.017 and +0.20, P = 0.009, respectively) compared to the NAM trial.
While no significant interaction between factors was detected in the other results for able-bodied subjects, algorithm type was found to have a significant effect on CT, PE and TP (F 3,27 = 13.494,P < 0.001, F 3,27 = 8.621, P < 0.001 and F 3,27 = 8.037, P = 0.001, respectively) (Fig. 5).Post-SAM had significantly lower CT (−1.09 s, P = 0.011) and higher PE and TP (+6.10%,P = 0.035 and +0.072 bits/s, P = 0.022, respectively) compared to NAM.Such significant improvements can be attributed to the supervised nature of the SAM algorithm allowing targeted restoration of intuitive control.
2) Amputee Subjects: For amputee subjects, UAM similarly mitigated the performance reduction from electrode perturbations.For AM1, post-UAM trials achieved +0.13 higher CR compared to NAM in both electrode-shifted states and in the medial-shifted case, performance was restored to be on par with that of the unshifted condition.An improvement was also seen in all other metrics, except for TP which was slightly lower for both post-UAM and post-SAM (−0.02 and −0.03 bits/s, respectively) in the lateral-shifted trials.With AM2, the adaptive controllers initially underperformed in the unshifted trials.However, this could be attributed to the particularly unintuitive mapping of DoF2 required for this subject (as described in Section II-C3).As such, the subject was still getting accustomed to the mapping after the initial familiarization trial.However, under the electrode-shifted conditions, CR in post-UAM trials closely matched that of the unshifted NAM trial, whereas NAM performance dropped markedly after electrode shift.Unlike the trend observed with able-bodied subjects, the benefits from UAM for amputee subjects, on average, exceeded those from SAM.Compared to post-SAM trials, +0.06 CR, −0.16 s in CT and +0.02 bits/s in TP was achieved in post-UAM trials although PE was lower at −1.81%.
3) Adaptation Analyses: To further examine the UAM's adaptation algorithm and its ability to compensate for sudden perturbations, we observed the original and shift-adapted basis vectors from AM1 (Fig. 6(a)).Adaptation to medial shifting resulted in basis vectors being more biased in the lateral direction and vice-versa.This self-adjustment of the controller's internal representations of motor task synergies in the sensor space, counteracted, in part, the physical displacement of the electrodes.As the forms of the synergy representations were still retained after adaptation, a familiar feed-forward translation of motor input is observed by the user.This property was also consistent with able-bodied subjects.Fig. 6(b) shows the average circular cross-correlation profiles between initial bases and their shift-adapted versions.Higher correlation is observed when the adapted bases are rotated in accordance to the physical direction to which they adapted for.This bias in the correlation profiles again indicate that the UAM-adapted synergies were similar to their original versions, albeit shifted in such a way as to counteract the electrode perturbation.
The general evolution of UAM and SAM models throughout an adaptation trial was quantified in terms of its disparity from the fully adapted version, which is the one deployed in post-UAM/SAM trials.This progress is illustrated in Fig. 7.For UAM, the model state after each target attempt was quantified by the absolute difference between normalized current and final basis vectors.For SAM, regression weight vectors were considered.Both algorithms are shown to exhibit stable model evolution as this model disparity is gradually decreasing.In other words, the control models stably converge to their final versions over the course of the adaptation trial.

B. Clothespin Relocation Tests
In the Clothespin Relocation Tests, stable co-adaptive control of a hand prosthesis via UAM was observed, even in trials with simulated electrode perturbations.Completion times of 22.48 s,  24.12 s and 25.12 s for the unshifted, lateral-trained and medialtrained cases, respectively, were achieved with no cases of dropped pins.For comparison, the test was also conducted using AM2's own device of 8 years which yielded 9.07 s in the fastest trial.Such a large difference in completion times was likely due to the subject's lack of familiarity with advanced myocontrol combined with the expertise gained from daily use of his own device.More importantly, UAM adaptation maintained intuitive control such that the multi-DoF functionalities of the advanced control method could be fully exploited, and the amount of postural compensation required by the user was significantly reduced, as shown by Fig. 8.In particular, the excessive lateral tilting of the trunk and internal rotation of the shoulder observed throughout trials involving the user's own device were absent in the trials employing UAM.A video recording of the trials can be found in the supplementary materials accompanying this article.

A. Unsupervised Adaptive Myocontrol
We have developed an adaptive myocontrol system that continuously learns from user input during real-time operation to facilitate an interface that is robust to the effects of electrode displacement, a significant and widely reported source of performance degradation in advanced myocontrol [17], [19], [35].In contrast to previous research outputs concerning real-time adaptation which have so far been constrained to supervised conditions [24]- [26] the solution presented here yielded comparable improvements via a label-free and non-disruptive adaptation scheme.The control model is initially trained from individual wrist DoF activations, for each of which a corresponding motor module is extracted.Simultaneous and proportional myocontrol is achieved by inferring motor command primitives from the reconstruction of live EMG input as the instantaneous combination of these motor modules [5].During real-time operation, system adaptation is initiated by the occurrence of conflicting command primitives which, in this case, is defined as the simultaneous activation of both positive and negative displacement in a DoF.While the co-activation of opposing (antagonistic) motor modules is inherent in natural limb control, for instance, when modulating joint stability, such commands do not translate smoothly to the regression-based position control scheme commonly employed in advanced myocontrol [5]- [8].In fact, this phenomenon can indicate a defective control model, caused either by incompatibilities of the initial synergies in reconstructing certain combined-DoF activations or from electrode displacements that misalign activation patterns in sensor space.By imposing constraints based on the intrinsic sparsity of motor module recruitment, stable adaptation of the control model can Fig. 8.The UAM algorithm was employed in the control of a commercial prosthesis capable of active grip and wrist rotation.With simultaneous and proportional control of these functions, an amputee subject (AM2) was capable of performing clothespin relocation tasks with minimal compensatory movements (2 nd row).This contrasts greatly with the degree of body compensation required when using the subject's own single function device, as indicated by the approximated angular deviation of the body centerline from the midline (1 st row).The benefits of the adaptive myocontrol scheme were demonstrated when the same tasks were repeated under simulated electrode displacement (3 rd and 4 rd row) where the high level of multi-function control was retained and compensatory movements remained minimal.
be driven purely by novel activation patterns presented during regular device operation.
As the practical advantages of unsupervised adaptation are well known, a few implementations have been recently introduced though these have yet to be verified in a real-timed closed-loop context [29], [30].Meanwhile, in the related field of body-machine interfacing, an adaptive controller based on autoencoding was shown to deliver superior target reaching performance over static decoding of upper limb kinematics [36].In general though, unsupervised system adaptation tends to be outperformed by supervised counterparts [20], [37].Indeed, this trend was observed here for able-bodied subjects where SAM consistently outperformed UAM in the virtual assessments (though this may be partially attributed to user learning as post-UAM trials precede post-SAM trials).This was expected given that the supervised nature of SAM allowed for highly targeted adjustments to its internal model.Interestingly, SAM's benefits were far less pronounced for amputee subjects who achieved better results in three out of the four performance metrics using UAM-adapted models (post-UAM trial) compared to SAM-adapted models (post-SAM trial).The lower effectiveness of SAM for amputee subjects may be explained by their reduced abilities to consistently recreate and maintain activation patterns due to reduced proprioceptive feedback [24], leading to mislabeling issues.With UAM, the algorithm is inherently immune to the effects of mislabeling due to its unsupervised nature and has shown to be a more suitable interface for the intended end users.

B. Compensating for Electrode Displacement
In the virtual target reaching trials, UAM achieved small performance improvements over the non-adaptive base model (NAM) under normative conditions.When electrode displacements were induced, the UAM adaptation scheme was capable of mitigating the resultant performance degradation, whereas relying solely on user compensation resulted in significant decrease of controller usability (Fig. 4).Under this controlled perturbation, system adaptation simultaneously complemented and alleviated the burden on user adaptation.By shifting model basis vectors to counteract the electrode displacements, users were not required to adopt drastically different control strategies as the overall shapes of synergy representations were consistently retained (Fig. 6), thus restoring intuitive control.

C. Stable Co-Adaptation Between User and Machine
In both the unsupervised and supervised adaptive schemes, the system and the user are simultaneously learning from one another.The model is adapted, as needed, in real-time while the user is also adjusting their control strategies according to the model changes.Thus, this is a two-learner scenario, where inappropriate design choices or parameters pertaining to the machine learner can lead to conflicts with the user's own learning, resulting in counter-effective or unstable adaptation [24], [38], [39].Here, stable system adaptation can be seen in Fig. 7 as the adapted model is gradually converging.This consistency in system adaptation supports the user's own adaptation as newly learned dynamics of the evolving system are transferable to its later stages.The same figure also indicates the relative magnitudes of model change when adapting under different cases of electrode perturbation.As expected, greater changes are made to the model with UAM under electrode-shifted conditions compared to the unshifted one.However, SAM resulted in roughly the same magnitudes of model change for electrode-shifted and unshifted adaptation trials.This suggests that relying solely on offline initialization yields sub-optimal models, even if done under ideally controlled conditions, and highlights the need for adaptive interfacing.
While this work demonstrates the benefits of co-adaptation, it does not quantify the user's own adaptation, as done in previous studies involving non-adaptive myocontrollers [18], [40].In such cases, the gradual reduction of control error over repetitions of a set task can be directly tied to motor adaptation.However, extending this analysis for multi-DoF adaptive controllers is non-trivial, especially as the influences of user and machine adaptation cannot be easily separated in performance metrics.By employing the classical experimental designs used to investigate visuomotor perturbations, the 'after effects' learning curve can be used to infer the magnitude of user adaptation [39].Furthermore, under the broader umbrella of human-machine interfacing, the brain-computer interface (BCI) domain has similarly explored co-adaptive methods to improve controller performance [41], [42].For example, in classification-based BCI, changes in control signal separability have been used as evidence for proficiency gain [43].Nonetheless, in this work, the simultaneous adaptation of user and machine is implicit, given that feedback from model changes are instantaneous and have a direct influence on the error-driven dynamics of the user's motor adaptation.

D. User Learning Effects
It is known that user learning leads to improved performance as subjects become more adept at advanced myocontrol [24], [25].As such, the first virtual target reaching trial was designated as a practice trial (Table I) to partially mitigate the influence of user learning in the assessment of different control algorithms.Indeed, performances in Trial 1 were noted to be consistently lower compared to those in Trial 6.In both trials the same NAM model was used, however, subjects would have had considerable practice with myocontrol by Trial 6, and while different algorithms will have different control dynamics, users will still have gained proficiencies in reproducing machine-separable activation patterns.To truly eliminate such learning effects from the study, one may repeat familiarization trials until increase in performance with NAM was no longer observed.However, this approach would not be feasible as different subjects would learn at different rates, resulting in a highly variable experiment runtime.Hence, the experimental protocol was designed such that the assessment of UAM and SAM preceded that of NAM in each condition of electrode perturbation.This was to ensure that any improved performance during trials employing the adaptive systems cannot be attributed to subjects learning to compensate.In fact, both adaptive systems outperformed NAM in the electrode-shifted trials despite subjects having more time to adapt to the perturbation by the time NAM was assessed (Fig. 4).

E. Clothespin Relocation Tests
Testing in virtual environments has already been demonstrated to correlate with performance in real-world settings [44].Furthermore, the performance metrics used in this study have been shown to be effective measures for distinguishing superior methods of control [34].Nonetheless, to demonstrate the applicability of UAM in the real-world, the algorithm was incorporated into the control of a multi-functional commercial hand prosthesis.This set-up allowed AM2 to perform clothespin relocation tasks using the UAM algorithm as the interface.The ability to concurrently and proportionally control both wrist rotation and grip aperture allowed the tasks to be performed in a natural manner.Moreover, robustness to electrode displacements was demonstrated when models initialized under electrode perturbed conditions were tested.With UAM adapting freely throughout the trials, similar completion times were achieved whilst postural compensations were minimal, leveraging the full advantages of the multi-functional prosthetic device.Conversely, when the same tasks were performed with the subject's own device, which was only capable of hand opening and closing, the amount of body compensation required was pronouncedly greater.Such requirements can be of detriment to users as repeated movement of joints outside their comfortable range can lead to fatigue or injury [45].It should be noted, however, that the subject's performance was far quicker when his own device was used.This was likely due to the high level of familiarity he has with his own device which he uses in day-to-day life, whereas exposure to advanced myocontrol had been minimal prior to the study.It is therefore reasonable to expect more comparable completion times given further familiarization with the co-adaptive interface.

F. Limitations and Future Work
In this work, we have focused only on electrode displacement as a source of signal non-stationarity.However, signal perturbations can stem from a wide variety of causes such as limb position, loading, perspiration and fatigue [46].While in theory, new synergy representations can be learned via UAM under any new condition, it is yet uncertain whether current design and parameter choices will produce a co-adaptive interface that is as effective in compensating for such disturbances to myoelectric interfacing.
Currently, the influence of new data in the UAM adaptation scheme is maintained via gradual decay of existing data.While this guarantees a responsive system adaptation, unlike SAM, no mechanism is currently in place to deal with scenarios where the learning algorithm is presented with highly repetitive activation patterns over an extended period.In such a case, the updated synergy representations may eventually become biased towards the reconstruction of only a subset of possible activation patterns.While the adaptation trigger implemented in this work has shown to be effective in regulating the data presented to the adaptation algorithm during virtual target reaching and clothespin relocation tasks, its suitability across the full range of activities of daily living has yet to be proven.Potentially, such concerns may be addressed by considering the novelty of candidate activation patterns to adapt from with regard to those recently presented to the adaptation algorithm [47].

V. CONCLUSION
In this work, a novel approach to enhancing usability and robustness in the control of advanced, multi-functional powered prostheses through the free and mutual co-adaptation of user and machine was presented.This development uniquely extends the incorporation of neuromuscular control concepts in humanmachine interfacing by leveraging known properties of muscle module recruitment to guide the adaptation of a myocontrol model, forgoing the need of explicit data labels.In virtual target reaching tasks, the proposed UAM system was capable of mitigating perturbation-induced performance degradation, without need for interruption in use, in both able-bodied and amputee subjects.In the case of amputee subjects, the increase in robustness exceeded even that of the more cumbersome supervised adaptive system.When integrated into a commercial hand prosthesis, UAM again demonstrated robustness to nonstationarities through continuous co-adaptation.During standardized Clothespin Relocation Tests, an end user completed the exercises with no discernable detriment despite electrode perturbations.Thus, the proposed system not only retains the functional benefits provided by supervised methods, but also brings improved robustness for end users while offering practical advantages for clinical deployment.To the authors' best knowledge, this demonstrates a new paradigm for co-adaptive myoelectric interfacing, where system learning is fully automated (self-initiated and unsupervised) and administered in real-time in concurrence with the user's own motor adaptation.

APPENDIX A DERIVATION OF UPDATE RULES FOR MODIFIED ONLINE NMF COST FUNCTION
The block-coordinate descent framework minimizes the cost function in (9) via alternating updates to the factor matrices (F l ) aj ← (F l ) aj − (η F ) aj ∂D k+l ∂ (F l ) aj (15) W k+l ia ← W k+l ia − (η W ) ia ∂D k+l ∂ W k+l ia (16) where η F and η W are learning rate matrices.The partial derivatives are obtained as follows: By choosing appropriate learning rates, the subtractive terms that arise via substitution in (15) and ( 16) can be eliminated to preserve non-negativity, yielding the update rules in (11) and (12).(20)

Fig. 1 .
Fig.1.Schematic for UAM integration in the virtual test environment.Motor command primitives are estimated from instantaneous muscle activation patterns based on the synergy-inspired control model and are translated to cursor co-ordinates in the virtual environment.Solid arrow lines represent continuous flows while the dashed line indicates that system adaptation only modifies control model parameters when toggled by excessively conflicting commands (opposing within-DoF primitives).

Fig. 2 .
Fig. 2. 2D task space used for the target reaching exercise.The current target is shown as a magenta solid-lined circle while dotted lines indicate other target locations.The spatial distribution of targets ensure thorough assessment of the multi-functional and proportional control capabilities of each myocontrol algorithm.

Fig. 3 .
Fig. 3. (a) The directions of transversal electrode shift.In lateralshifted trials the sensor band is rotated such that the reference pod (blue LED) is 1.2 cm closer to the lateral epicondyle.In medial-shifted trials the sensor band is rotated from its original position in the opposite direction.(b) The custom socket housing the sensor band.Mounted distally is a commercial bionic hand with an active rotatable wrist.

Fig. 4 .
Fig. 4. Virtual experiment results.Bar plots show results from ablebodied subjects while results from amputee subjects (AM1 and AM2) are separately shown as diamonds and squares, respectively.

Fig. 6 .
Fig. 6.(a) Model adaptation data from AM1's virtual trials.Grey shaded bars represent the initial model bases while resultant basis vectors from the lateral-shifted and medial-shifted adaptation trials of UAM are overlaid as blue and red shaded bars, respectively.On the x-axis, 'R' indicates the channel from the reference pod while 'L-' and 'M-' indicate a channel's position relative to the reference pod in the lateral and medial directions, respectively.(b)The cross-correlation between shift-adapted basis vectors and their original states, averaged over all subjects.The x-axis relates to the level of channel shift in the lateral, 'L-', and medial, 'M-', directions and the auto cross-correlation of base model synergies are shown as grey curves for reference.As the channels are physically distributed in a radial manner, circular cross-correlation is performed here where one vector is shifted circularly.

Fig. 7 .
Fig. 7. Adaptation progression with UAM (left) and SAM (right) during virtual target reaching trials.The x-axes indicate adaptation trial progress in terms of targets presented.Y-values are computed as the difference between the model at each stage of the adaptation trial and the final model at the end of the trial.For UAM, this means the comparison of synergy representations (the columns of W) with their end-of-trial versions.Shaded areas indicate the standard deviations.