Trajectory Control–An Effective Strategy for Controlling Multi-DOF Upper Limb Prosthetic Devices

Despite great innovations in upper-extremity prosthetic hardware in recent decades, controlling a multiple joint upper limb prosthesis such as an elbow/wrist/hand system is still an open clinical challenge, in large part due to an insufficient number of control inputs available to users. While simultaneous control is in its early stages, the common control approach is sequential control, in which joints and grasps are driven one at a time. In this paper, we introduce and evaluate a concept we call trajectory control, that builds upon this approach, in which motions of the wrist, elbow, and shoulder DOFs (and subsets of them) are coupled into predefined sets of coordinated trajectories; to be selected by the user and driven with a single input variable. These trajectories were designed based on an earlier motion study of activities of daily life obtained from human demonstrations. We experimentally evaluate the efficacy of our approach through a human subjects study in which tasks are performed in a virtual environment. The results show that as device complexity increased (i.e. greater number of DOFs corresponding to more proximal amputations), participants were able to complete tasks faster with trajectory control while exhibiting similar levels of body compensation when compared to sequential and simultaneous control. Additionally, participants found trajectory control to be more intuitive and displayed more natural movement.

. Subject performing a cup pouring task, seen wearing the HMD and is using the controller to operate the virtual wrist. In the top right a semi-transparent red cup is visible indicating the desired cup orientation goal, which turns green (bottom right) once the cup reaches the target. freedom (DOF) hands [1]. A lack of proper joint control can lead to unnatural movements [2], with many users developing overuse syndromes [3]. Partly due to a limited number of available control inputs and intuitive control strategies, little progress has been made with respect to the development of fully articulated prosthetic wrists and arm devices. Additionally, complex control strategies can induce additional cognitive burden that leads to an increase in device abandonment rates [4]. In this work we investigate a novel control approach and validate it by comparing it to other methods in virtual reality (VR) (Fig. 1).
Conventionally, myoelectric upper-limb prosthetic devices are operated with direct control, also known as sequential control. In multi-DOF prosthetic hands, a user is outfitted with a standard 2-site surface electromyography (sEMG) on their residual limb and uses a "patterning" of their muscle activity to select from a set of pre-programmed grasp types. After grasp selection, the user is then able to move all the joints of the hand simultaneously along the chosen grip motion path and open/close it with a single analog input. In transradial prosthetic devices that include a powered wrist rotator, sequential control can also be used to toggle between grasping modes and a pronation-supination mode. However, in fully articulated 3 DOF joints, such as the wrist and shoulder, controlling one joint at a time would be time consuming and poses a cognitive challenge in deciding the order in which the joints should be rotated. Some groups have leveraged This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ pattern recognition (PR) [5] to mitigate the need for tedious mode-switching by classifying muscle patterns to individual DOFs, for example, in shoulder disarticulate amputees that have undergone targeted muscle reinnervation [6]. On the other hand, simultaneous joint control, primarily explored in research, carries its own set of challenges, such as identifying enough available control inputs (e.g. sEMG sites).
Advancements in prostheses hardware is inadequate without an intuitive or practical way for amputees to control them [7], and various groups have attempted to bridge the gap through different interpretations of simultaneous control. For example, some groups have leveraged PR, where synergies between the residual limb and the device are coupled, such as the shoulder or elbow joint angles and the wrist in transradial amputees [8]. However, additional movement combinations would have to be introduced as additional classes, and thus increasing training time. A more "natural" approach could be through regression where data driven methods are used to map multiple inputs to multiple outputs simultaneous, as has been demonstrated in a 2 DOF prosthetic wrist [9]. These methods are robust under certain conditions, however, they are unlikely to reach the majority of prosthesis users as they can be either invasive, require additional hardware, or include very involved training and retraining phases.
Simultaneous controls have also been proposed without the use of ANNs, such as on-line optimization techniques in transhumeral amputees [10]. Wrist or elbow rotation could also be directly coupled to shoulder abduction [11]. Controlling a wrist device using symmetric or anti-symmetric mirroring of the healthy hand has also been shown to be useful for bimanual manipulations [12]. In shoulder disarticulate amputees, directly controlling the 3 DOF of the device's shoulder becomes less tractable, and solutions have included controlling the whole arm in the end-effector space using a foot interface [13], or, in the case of a wheelchair mounted robotic arm, a joystick [14]. These methods offer an additional control input beyond the standard two-site sEMG [15], and while they enable users to perform complex arm motions, they also impose a cognitive burden that limits efficacy.
In this paper we present a novel prosthesis control approach based on pre-programmed motion trajectories that allow all the joints of a prosthetic arm to be simultaneously controlled with a single input akin to sequential control. Our "trajectory control" method makes use of all available DOFs of a myoelectric prosthetic wrist, elbow-wrist, or shoulder-elbow-wrist device with the same standard clinical interfaces (e.g. 2-site sEMG), while also reducing the cognitive burden associated with controlling multiple joints simultaneously. Hereafter we refer to a wrist, elbow-wrist, and shoulder-elbow-wrist prosthesis as 3 DOF, 4 DOF, and 7 DOF devices, respectively, referring to the controllable DOF.
The prespecified motion trajectories are based on the authors' previous work on arm motions performing activities of daily living (ADL) (Fig. 2) [16]; in total there are 5 trajectories for the 3 DOF device, and 11 trajectories each for the 4 and 7 DOF devices. These trajectories were obtained by clustering and averaging arm motions from healthy participants performing an extensive set of ADL tasks. A range of participants were recorded in order to include individual motion (c) Each cluster was averaged to obtain a single representative trajectory, resulting in different number of prototypical motions for the 3, 4, and 7 DOF arm models.
variation [17]. We refer to the proposed approach as trajectory control, and we believe would offer users more intuitive, better movement cosmesis, and more assistive control while limiting body compensation. We test this hypothesis using various quantitative and qualitative evaluation metrics [18] that pin the proposed control against conventional myoelectric control methods that include sequential and simultaneous control.
We coin the term "movement cosmesis" to describe how natural the motion looks, analogous to conventional "cosmesis" that refers to how closely the prosthesis resembles a real human limb. It can be quantitatively estimated in various ways, such as identifying joint angle deviations from natural movement, or through generating smooth motions by minimizing jerk [19]. However, in this work we leverage the participants' own perception of movement cosmesis through a questionnaire.
The efficacy of trajectory control is demonstrated with participants without limb differences in VR (Fig. 1), specifically identifying whether leveraging trajectory control in 3, 4, and 7, DOF devices outperforms conventional control approaches (i.e. sequential and simultaneous). VR has been embraced across a wide range of applications [20] as a low-cost method to iterate and test new controls and device designs that extend to a real world setting [21]. Other applications of VR have included training amputees to use a new prosthesis [22], fitting prostheses [23], and rehabilitation [24]. This paper builds upon the authors' previous work, where learned prototypical motions were first developed [16] and tested in a pilot [25], by looking to demonstrate trajectory control as a feasible control method to position and orient the end-effector of various prosthetic devices. A summary of the method that was used to obtain prorotypical motions is included in section II. A; a full description can be found in [16]. The pilot study [25] indicated that trajectory control was superior to both sequential and simultaneous control on a number of criteria, however, the protocol primarily examined the execution of individual submovements rather than whole tasks, grasping was not required, and testing was limited to a 3 DOF device. In the present work we 1) expand the analysis to a more comprehensive task protocol where tasks are comprised of multiple submovements, 2) analyze 4 DOF and 7 DOF devices, and 3) address the following questions regarding trajectory control use in prosthetic devices: i ) does it reduce the time it takes to position and orient the hand, ii) does it help mitigate body compensation, and iii) are there differences in user preference between control methods. For the remainder of this paper, we begin with describing the data collection protocol and set up (section II). We proceed with the analyzing the results (section III) and follow with a discussion covering conclusions and future work (section IV).

II. EXPERIMENT PROTOCOL
The role of an upper limb prosthesis is primarily to restore independent living by enabling amputees perform ADL tasks. Therefore, participants were asked to complete a series of ADL tasks using various virtual prosthetic arms and controls. Body tracking and VR display architecture was designed after [23]. Twelve right-handed participants (ages 18-53) completed the study over one session, lasting from 6 to 8 hours. In addition to lunch, participants were encouraged to take as many breaks as they needed. Data processing and analysis was performed in MATLAB 2021a. This study protocol was approved by Yale University Institutional Review Board, HSC# 1610018511.

A. Overview of Prototypical Motions
Given that arm motion behavior likely occurs in a continuous and smooth manifold [26], we hypothesize that by clustering and extracting a limited number of prototypical arm motions we can enable prosthesis users to carry out a vast number of activities. Specifically, each task can be divided into generalizable subtasks, such as reaching or transferring. For example, the task of drinking from a cup would be completed using the following prototypical motions: reaching in front and bringing to mouth motions. From a user's perspective, they would toggle to an appropriate prototypical motion, execute it, then proceed to toggle to the next. Though the motions themselves are not flexible and could result in certain users being unable to complete certain subtaks, we nonetheless hope to demonstrate the potential of this novel approach.
These motions were extracted in [16] using a series of data driven methods, where the goal was to obtain a categorization of motions representative of the range of ADL that are directly relevant to prosthesis users. These included tasks related to hygiene, feeding, and transferring; for example, reaching the axilla, eating with a fork, and bimanual transferring of a box. The procedure to obtain the prototypical motions is summarized in Fig. 2. First, the arm joint angles of 12 healthy participants, including left and right handed, and different from those participating in the VR portion of this study, performing 25 ADL were collected in the real world in a laboratory setting. Each trial was then manually segmented into clear reaching and transferring sub-movements. Three repetitions for each participant were averaged using dynamic time warping (DTW) and DTW barycenter averaging (DBA) [27]. The motions were clustered using agglomerative Hierarchical Clustering [28], the distance metric was Ward's linkage using DTW [29], and the number of clusters was defined using the L-method [30]. Finally, a representative motion from each cluster was obtained using DBA, resulting in 5 unique motions for a 3 DOF model and 11 unique motions for the 4 DOF and 7 DOF models.

B. Virtual Reality Set Up
The VRE was designed in Unity, offering unimpaired participants an immersive environment to operate prosthetic arms akin to prosthesis users. Virtual objects were either designed in the environment, in Maya 3D modeling software, or downloaded from the web. Visual feedback was designed to imitate a possible set up in real life: a virtual computer screen displayed the mode and controls that the device was currently in, the control inputs coming from the user, and two posters placed on the left and right walls of the virtual room listing the modes for the participants to reference (Fig. 3, left panels); top lists prototypical motions for trajectory control and the bottom lists the order of arm joints. All segments of the prosthesis (humerus, forearm, and hand) were scaled to the average human arm dimensions. For simplicity, we only simulated right-arm prostheses, and therefore the VR environment was also designed for right handed participants. Although grasping is included in the protocol, given that it is not the focus, in-hand object kinematics and dynamics were not considered; successful initiation of a grasp (and likewise release) resulted in the object to automatically connect to the palm of the end-effector as if they had grasped it using their own hand.

C. Control Inputs
The control input for the virtual prosthesis was either the position and orientation of the right hand for the natural and the no-wrist control modes, or K-Mix (Keith McMillen Instruments, Berkeley CA, USA) (Fig. 4), a modular control board MIDI audio interface, operated mostly by the left hand for controls mimicking myoelectric prosthetics. The MIDI controller also ensured that the inputs were consistent across the different devices and control conditions, i.e. compared to the different button interfaces on a game controller, MIDI sliders are all operated the same way. To imitate sEMG inputs found in myoelectric devices, slider intensity corresponded to a velocity input; the right side of the slider was the "forward" direction and the left was "backward", while buttons were used (in sequential and trajectory control only) as toggles between grasping or switching between joints or prototypical motions. For each slider, one inch in the middle of each slider was defined as a "dead zone", where velocity input remained at zero, and whose value gradually increased towards the ends either in the positive or negative direction. This way participants could rest their finger on the slider without inducing motion; a particularly important feature for simultaneous control when multiple sliders were being used. A digital keyboard was selected over an analog due to the ability to program a "zeroing" of the sliders when fingers were no longer making contact, akin to a joystick springing back to the middle when released. The keyboard was set on the torso pad worn by the participants and is meant to be controlled with the left hand for the 3 and 4 DOF device, freeing up the right arm to move. For 7 DOF device, both hands were free to control the device since all the arm joints were controlled with the keyboard (Fig. 5).
In this study we used 12 Bonita motion capture cameras (Vicon, Oxford, UK), recording at 100 Hz with an accuracy within 1 mm and 1 • of orientation, to track the subjects' body segments: hand, forearm, humerus, torso, and pelvis. For easy donning, markers were placed on wearables. To track the hand markers were placed on a small pad with elastic straps. An orthopedic elbow brace (Orthomen ROM Elbow Brace, Foothill Ranch CA, USA) was used to track the forearm and humerus; the range of motion of the elbow joint was left unrestricted; though participants found it helpful to lock the elbow in a single configuration when keyboard was used to control it. A modified lacrosse pad (Gait Gunnar Lacrosse Shoulder Pads) was used to track the torso. Finally, a nylon tactical belt with a plastic buckle was used for pelvis tracking. Marker clusters for the torso and the pelvis were placed on the dorsal side to avoid optical occlusions (Fig. 5).
For prosthetic devices related to more proximal levels of amputation, distal segments were unused, for example, the hand and forearm were not tracked when the participants operated the elbow-wrist prosthetic device; the input came directly from the controller for those joints. The reference frame for the virtual device came from the most distal intact segment, for example, the humerus was used to position the base of the elbow-wrist device. For the 7 DOF device, the torso was used as reference frame. In addition to providing participants an immersive experience by displaying a virtual arm in place of their own, the presence of arm segments also acted as a reference for the positions of the prosthetic joints. The reflective marker locations include bony landmarks that were also used to calculate the joint coordinate reference frames according to [31]. Head tracking was performed through Vive (HTC, Taoyuan City, Taiwan) head mounted display (HMD). Calibration between Unity, Vicon, and Vive was performed automatically by minimizing the distance between a set of reflective markers and Vive joysticks.

D. Control Modes
Each participant completed the experiment using five unique control methods. Initially, participants completed tasks using a positive and a negative control, hereby referred to as natural and no-wrist controls, respectively. In natural control, participants used their own unencumbered hand to complete the tasks; this served as a performance benchmark that participants aspired to achieve when using the prostheses. With the aid of a marker cluster placed on the back of the hand, the virtual hand closely matched the position and orientation of the real hand.
The no-wrist control was a transradial test condition in which the virtual prosthesis lacked wrist mobility. The subjects' virtual hand was fixed in place relative to the forearm. In this mode, participants were to complete tasks without the ability to rotate their wrist, thereby needing to compensate using their residual limb and torso. For both the natural and no-wrist trials only, grasping was instantaneously initiated when the hand reached the target location and orientation.
Negative controls in which the virtual prosthesis lacked elbow or shoulder control, in addition to the wrist, were not included in the testing protocol due to the inability to perform most tasks; these limitations were identified during a pilot study.
Sequential control imitated current myoelectric technologies where users have access to only two control inputs. In this mode only one slider was in use, while switching was relegated to a button, mimicking the mode switching in myoelectric devices. While this set up seems distinct to the two-site sEMG input in myoelectric devices, functionally they are the same: each half of the slider rotates a joint either forward or backward. A single button press toggled the control mode down the joint list, a double tap toggled the mode up the list, and a prolonged 1 second press of the button toggled grasping (or releasing). Switching between controllable joints cycles in the following order: i) shoulder plane of elevation, ii) elevation, iii) shoulder internal rotation, iv) elbow flexionextension, v) wrist pronation-supination, vi) flexion-extension, and vii) radial-ulnar deviation. Only the relevant joints are included for the 3 and 4 DOF devices. This sequence was listed on a poster in the VRE that the participants could reference at any time (Fig. 3, bottom left panel). This switching scheme is akin to switching between grasps and available joints in current prostheses.
In simultaneous control, participants were granted access to all controllable joints at the same time by leveraging multiple sliders on the MIDI keyboard. Like natural control, this mode represents the state of the art and a theoretical condition where users have all control inputs available to them simultaneously; presently impractical with standard sEMG control which would require 2 × 8 inputs for 7 DOF devices (7 pairs of inputs for the joints and 1 pair for grasping). Each slider corresponded to a single DOF, and participants could decide whether they wanted to operate them using several fingers concurrently or one at a time. Much like each of the controllable joints, grasping and releasing was also performed using a dedicated slider.
Finally, in trajectory control participants had the same interface as the sequential control: one slider to move forward and backward along a prototypical arm motion, and a button to toggle between the modes and initiate a grasp. Implementing the learned motions from [16] there were 5, 11, and 11 options for the 3, 4, and 7 DOF devices, respectively; an example of each is displayed in Fig. 6. Each trajectory mode was listed on a poster in the VRE (Fig. 3, top left panel) accompanied by a list of tasks from which the motions were extracted from and would likely be the best candidates to help complete those tasks. In practice, we suspect users will have visual cues on hand, such as a "cheat sheet", as they become accustomed to trajectory control, though this warrants its own investigation beyond the scope of this work. Although individual trajectories were composed using coordinated joint motions, it was useful to define them at an end-point level; in contrast to sequential and simultaneous controls which are best defined at a joint level. For all the control modes, a virtual computer screen was placed in front of the subjects while they completed tasks, displaying the present mode and their current input.

E. Study Procedure
The protocol included a set of tasks inspired by work done in evaluating upper-limb prosthesis performance [32]. Because motions in trajectory control were generated by averaging a collection of diverse tasks and participants, there is no guarantee that any individual motion would be able to successfully complete any of those tasks, and therefore a comprehensive demonstration was deemed necessary. We selected a set of 12 ADL tasks that required the use of only the right arm, covered a variety of locations, and included both reaching motion and object manipulation (Table I). These were chosen to span as many of the learned motion modes as possible. 9 tasks were chosen from those used in generating the prototypical motions and were set up according to the same dimensions listed in [5]. 3 totally novel tasks that are comprised of a series of distinct submovements were also selected to test trajectory control with tasks that are significantly different from any task that was used to learn the motions. The additional tasks include the following: pouring from a kettle, turning a screw, and cooking (Fig. 3). The 12 tasks were prioritized previously for hand function assessment protocols [33] and amputee surveys [34].
The tasks were completed using each control mode in the order depicted in Fig. 7: the participants completed the series of 12 tasks using each control mode starting with the natural condition, followed by no-wrist control, then in a randomized control condition order they completed the tasks with 3 DOF device, followed by the 4 DOF and 7 DOF devices. The tasks were randomized for each series. Prior to each new device and control condition, participants were given a general overview of the controls, which included a description of the trajectories and the control order of the operable joints.
Participants were also provided unstructured practice time to control the device itself prior to each task to mitigate learning effects when faced with an unfamiliar environment or control interface. Training ended when participants felt ready and had a strategy for how they were going to accomplish the task, such that they were not hesitating when recording started. For  trajectory and sequential control, participants preselected the mode during training with which they would start the trial. Repeating tasks was permitted, as we hoped to capture the participants' best efforts. Although performance was closely monitored by the experimenters, participants were additionally instructed to indicate if a task could not be completed.
Prior to the start of each task, participants were asked to begin with their hands relaxed by their side if standing or on their laps if sitting. They were then instructed to match the position and orientation of the end-effector, indicated with a red semi-transparent hand model (Fig. 8). For tasks that included several subtasks, the successful completion of a subtask automatically initiated the next one. Each subtask included a goal and tolerance for both the position and orientation and a hand closure requirement; generally, within 2 centimeters and 10 • , established through pilot studies that we conducted. The task goals were defined independently from the prototypical motions that make up trajectory control to qualify the method's performance. A successful completion of a grasping subtask resulted in the object to automatically be placed within the hand (i.e. initiating a grasp while in the pose seen in Fig. 8b) and was followed by a transferring subtask in which the object, rather than the end-effector, was to be placed in a new location and orientation starting from where they grasped it. For sequential and simultaneous control, the controllable joints were reset to zero prior to the start of a task. To ensure that participants intentionally completed the tasks, the target pose had to be held for one second.
Several conditions were implemented to ensure that the virtual tasks would be representative of the real world. As in real life, if a cup was tilted beyond 180 degrees during a transferring phase, then the whole task was considered a "fail"; this condition is unique to the cup drinking and the cup pouring tasks. Screwing and turning a knob would likewise fail if the end-effector did not maintain its position while rotating. Finally, the whole task was flagged as incomplete if the participant was unable to complete each subtask within one minute; no cutoff time was included for whole tasks.

F. Data Analysis
During the experiment the following were captured at a rate of 50 Hz: position and orientation of each body segment, inputs from the MIDI controller, and task success/failure status. The start of the trial was manually detected by the researchers when participants began to move, while the end was automatically determined by the software when the target pose for the final subtask was reached and held for 1 second. A moving average of window size 5 was used to smooth the data after large discontinuities were manually removed.
Assessment of control methods relative to one another was done using quantitative and qualitative metrics. These included evaluating the time it took to complete each task, and range of motion (ROM) and Cartesian path lengths of each joint as indicators of body compensation. These have generally been the only quantitative indicators of compensatory movement [35], where deviations from the normal range can indicate either an inability to move certain joints, or extraneous motions that place increased burden on the joints [3]. For example, when pouring a liquid, the proper end-effector motion can be achieved either through pronation of the forearm or by compensating through the elevation of the elbow; this is in part what inspired the implementation of a synergy between the two joints in a prosthesis [11]. Consider the challenges associated with orienting the end-effector when reaching objects on the table in various configurations without the ability to supinate the wrist. Coordinate systems for the arm joints and torso were established according to [31]. Using pelvis as the reference frame, torso angles are described in the following order: flexion-extension (bowing forward and back), leaning left-right, and turning left-right (twisting), where extension, leaning right, and turning left are positive directions. Cartesian path length of each segment, L, was calculated as a sum of Euclidean distances between globally defined sampled points, where X, Y , and Z correspond to the three Cartesian components of a trajectory in space and j is the total number of sample points in that trial.
Several processing steps were taken to ensure interpretable results. According to the shoulder angle definitions [36], a discontinuity appears when the humerus is perfectly vertical, aligning the plane of elevation and internal rotation axes. We therefore ignore the plane of elevation and internal rotation data when calculating ROM or Cartesian path length when humeral elevation is below 15 degrees.
At the end of the experiment participants were asked to fill out a survey that included questions to gauge their perception of each of the control methods. First their overall preference was assessed on a scale of 1 (did not like) to 5 (very much liked). Then, participants were also asked to indicate on a scale of 1 (disagree) to 5 (agree) to the following statements about each control-device combination: i ) easy to learn, ii) appeared natural (did the motion resemble a healthy arm), iii) mentally challenging (gauges the cognitive burden), and iv) physically challenging (gauges the amount of body compensation).
Before running statistical comparisons, missing data due to failed tasks was estimated using Multiple Imputation method with Monte Carlo Markov Chain; done in SPSS 2019. Tasks that were not completed by any participants with a given device and control condition cannot be imputed and were therefore omitted from the analysis for the other control conditions as well. Multivariate analysis of variance (MANOVA) was performed for each device to test if there was a significant difference between controls (excluding natural and no-wrist conditions) while accounting for the variability of several factors simultaneously, namely the differences between participants and tasks. The α-level was adjusted to 0.0031 using Bonferroni correction to account for repeated MANOVA calculations; 16 tests in total were made for each device. Follow up paired t-tests were performed between pairs of controls for individual factors. Repeated testing was adjusted with Bonferroni correction as well, accounting for tests within each dependent variable category (i.e. time, Cartesian path length, and ROM). Because the order of the 3, 4, and 7 DOF devices were not randomized in the experiment protocol, quantitative and qualitative assessment comparisons were not made between them but within each device category, e.g. the 4 DOF trajectory control was only compared to the other two 4 DOF controls.

III. RESULTS
All participants completed the experiment to the best of their ability, though most were not able to complete every task with every control mode. Reasons for failing to complete a task included timing out, failing a transferring condition, or quitting. Some tasks under certain control conditions were not completed by any participant and included reach to axilla and the eating with a spoon tasks using the no-wrist control, reach to axilla task with the 3 DOF trajectory control, and cooking with the 7 DOF sequential control. A detailed list of which modes lead to the most failed tasks is shown in Fig. 9.
Comparison between the time to complete the tasks, Cartesian path lengths, and ROM were made between pairs of control conditions. Approximately 7.4% of the distributions being compared did not strictly meet the assumption of normality under the Shapiro-Wilk test. The data across all conditions did appear to come from a normal distribution, so the analyses proceed.
After accounting for the variability between subjects and tasks, time was found to be significantly different (p < 0.001, MANOVA) between prosthesis control conditions for both 4 and 7 DOF devices; it was not significant for 3 DOF devices, with p-value = 0.86. The time it took each participant to complete the entire experiment is displayed in Fig. 9, along with the number of mode switching that occurred when using sequential and trajectory controls and the total number of failed tasks. A representative completion time for each test condition was calculated by summing the average task times across participants. Colored bars in Fig. 9 and Fig. 10 do not include tasks that participants were not able to complete for any single control condition, such that they contain the same data, but were included in the dark bars for the modes where they were completed by at least one participant.
After accounting for the variability between subjects and tasks, Cartesian path length was found to be significantly different in the shoulder for the 7 DOF devices (p = 0.03) and in the wrist and elbow for both the 4 DOF (p = 0.002 and p = 0.02, respectively) and 7 DOF (p < 0.001 and p = 0.01, respective) devices. Although significance was found for the wrist and elbow between the 4 DOF devices when using MANOVA, significant difference could not be established when comparing individual pairs of controls using a paired t-test. Representative Cartesian path length of each joint center is summarized in Fig. 10. Like the completion times, these were calculated by summing the average lengths per task for each control condition. Distal joints on average travelled further than proximal ones. Thus, differences between control modes increase in the distal joints, an observation that can be seen to further increase the more DOF were controlled. For the 7 DOF device, we observe that the dark bars in simultaneous and trajectory controls are nonetheless shorter Fig. 9. Experiment completion times are displayed per subject and per control. Colored bars all contain the same data, while dark bars may contain tasks that other control modes do not include. Each participants' total time spent using each control mode and device is displayed as a dot. Paired-test results (between colored bars) are displayed between pairs of controls for each device; significance levels were set at p-values of 0.05, 0.01, and 0.001 after accounting for repeated testing. Numbers below the bar charts indicate the number of mode switching that occurred and the number of failed tasks, whether per subject or per control mode. Fig. 10. Representative Cartesian path lengths are displayed as colored and dark bars. Colored bars all contain the same data, while dark bars may contain tasks that other control modes do not include. Each participants' total Cartesian path length for each control mode and device is displayed as a dot. Paired t-test results are displayed between pairs of controls for each device; significance levels were set at p-values of 0.05, 0.01, and 0.001 after accounting for repeated testing.
than the colored bar in sequential, pointing to the inefficiency of sequential control.
Significant differences in ROM between control conditions are summarized in Table II: using MANOVA, p-values are calculated to test the significance of the difference between control methods for each device and joint angle. ROM of each joint angle was assessed to evaluate body compensation under different conditions and control modes by calculating the average ROM across tasks and participants (Fig. 11). For the 3 DOF condition, the no-wrist control generally had the largest torso, shoulder, and elbow ROM, while the natural control had the highest wrist ROM and the lowest ROM in the other joints, satisfying the control expectations. Significance levels are included, though only the 4 DOF device had a significant difference between two conditions related to a body joint (rather than a prosthesis joint), occurring between simultaneous and trajectory controls. The torso, shoulder, and elbow ROM were generally mixed between the three prosthesis control conditions. While ROM is primarily analyzed with respect to body compensation, it can also be used to evaluate device usage as it relates to power consumption and motion efficiency.
After each experiment session subjects had the opportunity to give verbal feedback as well as rank the controls on various qualities on a scale from 1 to 5 (Fig. 12). Between the three prosthesis controls, there appear to be several trends that gradually change from 3 to 4 DOF and 4 to 7 DOF devices. For example, the preference given to trajectory control appear to improve relative to the other modes the more DOF a device had. Note that the scores reflect the participants' perceptions, so survey results may end up being inconsistent with the

IV. DISCUSSION
In this paper several upper-limb prosthesis control methods were assessed to highlight the proposed trajectory control. Assessments included evaluating the time it took participants to complete the tasks, body compensation, personal preferences, and the cognitive burden that users experienced when faced with complex orienting tasks. No technical issues were reported while achieving the end-effector goal location and orientation, and although some struggled to complete certain tasks, participants noted that the tolerances were fair.  Participants consistently elected to use the available control method to operate the prosthesis when completing tasks despite not being explicitly required to; they could have left the prosthesis in its initial configuration and attempt to complete the task without moving its joints. Even for the wrist device, when participants could theoretically complete some tasks by only compensating with their own elbows and shoulders, all controllable joints were in use, even at the cost of longer completion times. Ensuring that participants were not feeling pressured to operate the joints, they reassured that it was entirely because they wanted to mitigate body compensation.
Simultaneous control was not used as expected, and participants largely operated the DOF sequentially. The difficulty associated with visualizing the interaction between the different DOF was highly prohibitive from their seamless and simultaneous operation. Task completion was nonetheless faster than using sequential control, and likely due to the challenges associated with switching [37]. This contributed to participants identifying this mode, as well as trajectory, as easier to operate, and generally preferred over sequential. This is consistent with previous findings [38]. While trajectory control included a switching functionality to change motions, it was not always necessary, since a desired motion mode was preselected during training and could potentially be all that a participant would need during the task. This demonstrates that positioning and orienting the hand using learned motions is not only feasible, but potentially superior under certain conditions to the other control methods.
When it comes to completing the tasks in a natural way with low extraneous movements and at a timely manner, there appears to be a tradeoff. One interesting finding was that the no-wrist control was significantly faster than the other 3 DOF controls. We suspect it was due to the reduced cognitive burden and likely due to the simplicity of the orienting strategy; participants had a clear expectation of what was possible, and thus vastly simplified their control plan. However, it also had the largest amount of body compensation, appeared unnatural, and was the least preferred by all participants, suggesting that movement cosmesis and reduced body compensation are likely more important than speed alone.
The time it took to complete the experiment with each of the control modes was statistically indistinguishable for the 3 DOF device, likely because positioning the end-effector was mostly the same while the controls were only be used to set the orientation. For 4 and 7 DOF devices, simultaneous control was generally faster than sequential, mainly due to no longer needing to laboriously switch between joint controls. However, despite involving switching, trajectory control was even faster, only requiring users to operate a single slider that automatically oriented the end-effector close to the desired goal. Given that the prototypical motions were generated using a much larger set of motions, we did not expect the final hand orientation to be exact. However, body compensation, measured as ROM and Cartesian path length, was largely comparable to the other controls and had the capacity to re-orient the hand. This suggests that the reduced cognitive burden may outweigh a reduction in direct orientation control.
Training time was not quantified, but certain observations were made. During natural and no-wrist control, training time was unsurprisingly very short, where participants primarily focused on learning what hand positions and orientations satisfied the target conditions. Training time during sequential and simultaneous controls was spent learning how the joints move and interact, while during trajectory control training time was used to learn and see what the motions were. However, as participants moved from the 3 to the 4 DOF device the training time somewhat shortened for the sequential and simultaneous controls and lengthened for trajectory control. When controlling the joints independently, the addition of an elbow did not impose much more complexity and was largely akin to the 3 DOF controls, whereas the participants had to relearn a brand-new set of motions for trajectory control. The opposite was observed for the 7 DOF device. Relearning a new set of motions for trajectory control was not only similar, but participants noted that they were more intuitive than the 4 DOF motions given the more straightforward association between whole arm motions and tasks. The same was not true for controlling the joints independently, and the addition of three new controllable DOF (i.e. shoulder), was significantly more difficult to intuitively grasp than the distal joints. However, when using sequential and simultaneous control, participants seldom referred to the virtual posters. The opposite was true when using trajectory control, where participants referenced the poster almost every time they switched modes. Training time appeared to have been relegated to the poster, so a more in-depth investigation should address learning and memorization of the motions in trajectory control.
It must be acknowledged that the preselection of modes during training could bias the time it would normally take to complete tasks with sequential or trajectory controls in a real-world setting. However, this was necessary to maintain consistency between tasks, as well as to demonstrate the best-case scenario for each control method.
Participant feedback confirmed several notable observations. Those that had an easier time controlling the devices at a joint level, normally preferred the sequential and simultaneous controls, though most participants preferred trajectory the more DOF were being controlled. For the 7 DOF device, sequential control was preferred by some over simultaneous, noting that simultaneous control had too many inputs to keep track of, but on the other hand, the excessive mode switching that was necessitated in sequential control was cumbersome. Most participants preferred trajectory control in the 7 DOF device, as it combined the simplicity of controlling one DOF at a time and simultaneous joint movement. Furthermore, because prosthesis rejection rates appear to be higher for more proximal levels of amputation [4], we believe that the application of this work will likely have the most impact in the elbow-wrist, and shoulder-elbow-wrist prosthetic devices. With certain improvements to trajectory control, such as adding an adaptive feature to the end-effector position to be able to complete all tasks, we hope to increase prosthesis user acceptance rates.
The learned motions in trajectory control were effective at carrying out the tasks they were generated from, by demonstrating that participants were able to complete most of the tasks without inducing further body compensation; seen as a lack of statistically significant differences between control methods. Furthermore, participants did not appear to have trouble visualizing which prototypical motions to use for the three novel tasks, for example, they were quick to realize that the doorknob motion was best for turning the stove knob. This suggests that proficient users would be able to extend the use of trajectories beyond the tasks that they were designed after. Moreover, these findings highlight the feasibility of using descriptive synergies (prototypical motions obtained from observing human motion) in prostheses over prescriptive synergies (lower dimensional synergistic control inputs) which are significantly more difficult to identify and implement [39]; it is unclear if there is a true neurological or muscular synergy. We suspect that the concept of using learned motions could be extended to other devices and sets of tasks, such as the JACO wheelchair-mounted robotic arm [14].

A. Future Work
Some improvements to the experimental design can be made. Both time and survey responses attempted to indirectly capture the cognitive burden associated with each of the control modes. However, cognitive burden could alternatively be measured directly using pupillometry, as has been demonstrated others fields, such as in prosthesis use [40] and driving [41].
While our experiment made certain simplifications, in the real world, dynamics play a major role in manipulation. In order to better simulate a real-world prosthesis use and recreate a more immersive experience, object interaction dynamics, akin to [21], will need to be considered in future efforts.
In our experiment, we deliberately avoided bimanual tasks due to the complexity associated with coordinating control with the healthy hand and a lack of haptic feedback. There has been a bias towards assessing unilateral tasks by the research community [42]. Given that the learned motions were generated in part by motions related to bimanual interactions [5], such as transferring a box, further validation of trajectory control could include bimanual tasks as well.
Improvements to the functionality of prosthetic devices need not to be limited to advancements in software. Alternative hardware decisions could simplify the types of controls that are needed altogether, for example, by setting one of the wrist DOF to an alternative axis or trajectory. An example of this includes a single DOF wrist device that rotates around an oblique axis [43]; one of the prototypical wrist motions encompasses this rotation, namely the flexion/deviation mode.
As a proxy to sEMG control inputs, the keyboard was used to assess prosthesis control. While for sequential and trajectory control the input is a single slider, simultaneous control involved up to eight sliders. This interface involved a degree of finger dexterity that will unlikely be present in a marketable device. As an imperfect imitation of sEMG, it introduces error. While we hope the comparison stands, further improvements to the cognitive burden and robustness of operating several DOF simultaneously [6], [13] could ultimately outweigh the simplicity of trajectory control. Finally, it is likely that endpoint control is what humans use to program reaching and grasping movements, so the idea of using a predefined decoupled set of locations and orientations, rather than joint motions, could be a possible future direction [44]. In the meantime, trajectory control highlights the benefits of using learned motions and suggests that a complex, yet practical, solution will likely be a semi-autonomous one and where additional trajectories can be added by the users.