FlyJacket: An Upper Body Soft Exoskeleton for Immersive Drone Control

Most human–drone interfaces, such as joysticks and remote controllers, require attention and developed skills during teleoperation. Wearable interfaces could enable a more natural and intuitive control of drones, which would make this technology accessible to a larger population of users. In this letter, we describe a soft exoskeleton, so called FlyJacket, designed for naïve users that want to control a drone with upper body gestures in an intuitive manner. The exoskeleton includes a motion-tracking device to monitor body movements, an arm support system to prevent fatigue, and is coupled to goggles for first-person-view from the drone perspective. Tests were performed with participants flying a simulated fixed-wing drone moving at a constant speed; participants’ performance was more consistent when using the FlyJacket with the arm support than when performing the same task with a remote controller. Furthermore, participants felt more immersed, had more sensation of flying, and reported less fatigue when the arm support was enabled. The FlyJacket has been demonstrated for the teleoperation of a real drone.

environments because of their capability to extend human perception and range of action in unprecedented ways [8]. Fields of application include aerial mapping, agriculture, military applications, search-and-rescue, transportation and delivery [5]. In most of these fields, drones work cooperatively with users as they increase human space perception providing information that would not be available from a ground perspective [9], [10]. However, the use of joysticks or remote controllers for drone teleoperation is a non-intuitive and challenging task, which becomes cognitively demanding during long-term operations [11]. The development of more intuitive control interfaces could improve flight efficiency, reduce errors, and allow users to shift their attention from the task of control to the evaluation of the information provided by the drone.
Human-robot interfaces could be improved by focusing on natural human gestures captured by wearable sensors. Indeed, the use of wearable devices, such as exoskeletons, has been shown to enhance control intuitiveness and immersion [12], [13]. Over the last decade, the development of gesturebased interfaces for flight control have focused on the use of upper body movements to control the flight with two different approaches.
The first approach relies on moving platforms that can support the entire weight of the person. For example, using the Birdly (Somniacs SA, Zurich, Switzerland) [14] or the Hypersuit (Theory, Paris, France) [15], the person lies horizontally on the platform with the arms spread out on wing-like structures to control a simulated bird or wing-suit. These platforms make use of a virtual reality headset to provide a first-person view from the flying agent's perspective. Flight control is achieved by moving the hands or the arms. The wing-like structure supports the arms weight, preventing physical fatigue during long flight sessions. Although these platforms were designed as virtual reality devices for making people feel free to fly like a bird in virtual reality, we have shown that they can also be used for immersive control of a real drone [16]. Despite the impressive rendering of flight experiences, these platforms suffer from two drawbacks when it comes to drone control: the support structures tend to constrain the range of human gestures to very few degrees of freedom, such as wrist rotation, and they are bulky and heavy, which prevents their usability in real-world drone operation.
The second approach lets users command the drone with upper body movements recorded by external tracking systems [17]- [19]. Here the person is free to use a larger variety of gestures to control the drone, but the lack of body support can This work is licensed under a Creative Commons Attribution 3.0 License. For more information, see http://creativecommons.org/licenses/by/3.0/ cause fatigue. Furthermore, the necessity of external devices for tracking body motion limits the method to an indoor use. Yet, one study used EMG sensors on the forearm to control a drone [20]. For most of the cited studies, the control is carriedout from a third-person view or through a monitor, which limits the immersion.
In this work, we describe a different approach to wearable drone control that attempts to capture the best features of previous approaches. It consists of a sensorised upper-body soft exoskeleton, coupled with virtual reality or first-person view goggles, that maps torso movements into control instructions for a fixed-wing drone flying at a constant speed and equipped with a frontal camera. The exosuit, which we call FlyJacket, is equipped with unobtrusive and removable arm supports, which allow users to fly with their arms spread out without experiencing fatigue. This jacket integrates sensors for body motion detection and can easily fit into a backpack for rapid deployment in diverse situations, such as rescue missions, inspection, or personal leisure. For instance, Section IV and the complementary video describe an application for search-andrescue. The user flies a drone with the jacket and exploits the video feedback to identify points of interest, such as injured persons or hazards, and can tag them to create a georeferenced map for future interventions.

II. EXOSUIT DESIGN
The design of an upper body exosuit for gesture control requires three main challenges to be addressed. The first challenge is to ensure freedom of motion and adaptation to different morphologies and both genders. A second challenge is to efficiently exchange forces between the user and the exosuit which requires a tight interaction to prevent parasitic motions that may deteriorate body motion detection. Also, a tight interaction between the body and the wearable is required to avoid slippage between the skin and the device that may lead to discomfort or injuries. The exchange of forces from the exosuit to the human must be localized in areas of the body that can withstand high pressure. For example, body joints and the chest are very sensitive to pressure, while the arm and the pelvis can be heavily loaded without pain [21], [22]. The third and final challenge is to prevent users' fatigue caused by remaining in unusual or tiring postures for extended periods of time. Other desirable features include a small size and weight of the exosuit, the use of breathable materials to prevent sweating, and a design that allows users to easily put on and remove the exosuit without external aid.

A. Fabric Design
The first two challenges are addressed by combining a layer made of elastic fabric, shown in blue, with elements of inextensible fabric shown in black in Fig. 1(a). The softness and extensibility of the elastic layer, made of sport fabric (100% polyester mesh knit), allow the user's motion freedom and adaptation to different morphologies and genders. In addition, the intrinsic compliance of the soft material can compensate for misalignments between center of rotation of the human and the exoskeleton's joint. Such misalignments are a well-known problem for  [21] and [22]). rigid exoskeletons [23], [24] as even a small mismatch in kinematics can induce a moment on the human body, leading to injuries. As shown in Fig. 1, the shoulder joints, elbow joints, a space for breast and the lower part of the torso of the exosuit are made of elastic fabric. This elastic fabric is also breathable and can evacuate sweat during intense flight sessions. The inextensible elements made of leather are connected to the elastic layer with snap buttons. These inextensible elements provide strong anchoring areas to the body as they can be strongly tightened to these regions with adjustments in order to transmit forces without slippage between the body and the exosuit. The inextensible elements are located on the upper torso, the pelvis, the arms and forearms, which are regions that can tolerate moderate to heavy loads (see Fig. 1(b)) [21], [22]. The inextensible elements are equipped with adjustments, shown by green circles in Fig. 1(a) and magnified in Fig. 2(a)-(d), that allow preserving motion freedom and adaptability of the jacket to different morphologies and body dimensions (minimal and maximal dimensions are given in Fig. 2). To adjust the tightening at the arm and forearm, cable lacing with a Boa closure system (Boa Technology Inc. Denver, USA) allows adapting their diameter with only one hand (see Fig. 2(c) and (d)). The leather parts of the arms possess an elastic segment to allow changes in arm volume due to muscular contraction during motion (see magnification of Fig. 1(a)). At the upper shoulder and at the middle of the torso tightening can be adapted with an inextensible fabric band and ladder-lock buckles (see Fig. 2(a) and (b)). The inextensible chest band is in series with an elastic segment that provides some compliance during change in the torso circumference due to breathing. The middle of the chest is made of elastic fabric to allow extra room for the breast (see Fig. 2(b)). On the back of the belt there is an elastic pocket to store electronics (see Fig. 3(a)).

B. Arm Support
A previous study on naïve individuals showed that torso movements are a natural and immersive way of flying a fixedwing drone [25]. This study found that drone pitch can be intuitively controlled by bending the torso forward and backward in the sagittal plane and drone roll can be controlled by a combination of bending in the frontal plane and rotation along the longitudinal axis of the torso (see Fig. 4(e) and (f)). Even if with this flight style the arms were not used to directly control the drone attitude, during trials we observed that participants instinctively spread out their arms when flying with either the elbows straight (see Fig. 3(a)) or bent ( Fig. 3(b)). In this position, two forces are acting on the arms: the arm's weight (Wa) which induces adduction, and the forearm's weight (Wf) which also induces adduction and, if the elbow is bent, an internal rotation of the shoulder ( Fig. 3(b)). Keeping this position for extended periods of time is tiring as shoulder muscles must counteract the moments induced by these forces. Therefore, the challenge of keeping the arms in this tiring posture is tackled in the FlyJacket by adding two arm supports that prevent fatigue (see Fig. 3). Research on body support has gradually shifted from bulky and heavy multi-joint systems [24], [26] to lightweight and wearable soft devices. Some soft body supports use actuated cables anchored to body segments [27]- [29]. However, drawbacks of this approach are the large standoff needed to create sufficient torque and the compressive loads applied to the joints during actuation. Another approach is to use pneumatic pouches [30]- [32]. The main disadvantage of pneumatic devices is the need for a compressor for the actuation, which prevents portability. Additionally, support of multiple joints would require a complex design of pouches and valves, which is not practical.
Thus, in the FlyJacket, we chose to use a linear passive gas spring (Eckold AG, Switzerland) between the upper arm and hips (see Fig. 3) because this region of the body is the best suited to bear the high forces generated by the arm support [21] ( Fig. 1(b)). The main advantage of the proposed design is that it does not attempt to mimic the shoulder joint anatomy, such as exoskeletons in [12], [13], [26]; resulting in a simpler design without major reductions in the range of motion (ROM). The arm and forearm are supported by two plates made of 3D printed Acrytonitrite Butadiene Styrene (ABS) (see Fig. 3(d)) with small cushions inside for comfort. These plates are interconnected by two passive joints: a rotating joint made of ball bearings to bend the elbow and a linear joint to absorb misalignments between the center of rotation of the elbow and the rotating joint of the support (see Fig. 3(e)). The connection between the gas spring and the waist belt is made of a ball and socket joint (Fig. 3(c)). The connection to the arm plates consists of a hinge joint. This solution allows resisting to the internal shoulder rotation induced by the forearm's weight therefore preventing fatigue (see Fig. 3(b)). The ROM in shoulder internal rotation is almost fully constrained in order to provide shoulder stability, but still giving the possibility to the user to put and remove the virtual reality goggles by themselves. ROM in adduction/abduction movement is limited in order to support the arm's weight in resting position (δ = 40°, Fig. 3(a)) and because of the limited length of the gas spring (δ = 120 • ). The arm support plates can be easily disconnected from the gas springs, thanks to a connector made of magnets that facilitate the placement of the gas spring. This support is secured by a screw to resist strong acceleration (see magnification in Fig. 3(d)). Gas spring supports can be entirely removed from the jacket if the user prefers to use another flight style, for example with the arms along the body. Although the gas spring used in the experiments described here is passive and calibrated to provide arm buoyancy, the same design could be actuated by a linear motor to provide haptic feedback from the drone wings to the arm level.

C. Movement Tracking System
The full movement recording system is composed of a Transmission Unit to send commands to the drone, a Microcontroller Unit (MCU, STM32F100, STMicroelectronics, Geneva, Switzerland) for real-time processing located in the electronics pocket, and an Inertial Measurement Unit (IMU) (LSM9DS1, STMicroelectronics) located on the back of the leather element on the upper torso ( Fig. 3(a)). The IMU is connected to the MCU via a Serial Peripheral Interface (SPI). The IMU records  the motion and the MCU processes in real-time this information to send posture commands to the simulated drone on the PC or to a real drone. Then, the steady connection offered by the leather element that supports the IMU enables precise tracking of torso motion without interference from motion of other body parts, such as the arms. For this first study on the FlyJacket, a simple flight style using only the torso to control a drone was used. However, other flight styles can be implemented and additional IMUs can be inserted on the inextensible element on the upper arms, forearms or pelvis to record the motion of these body parts.

III. EXPERIMENTAL VALIDATION
In order to assess the exosuit for drone control and the effectiveness of the arm support system, 32 participants (25 men and 7 women, age 27.3 ± 5.9 years; mean ± SD) controlled a simulated fixed-wing drone either with torso movements wearing the FlyJacket, or with a remote controller (Hobbyking 6CH RC Flight Simulator System, Hobbyking, Kwun-Tong, Hong Kong). In both cases, they used virtual reality googles (Oculus Rift Development Kit 2, Oculus VR, Menlo Park, USA).
Each participant performed the experiment under only one flying condition. 24 participants used the FlyJacket in one of the three following conditions (8 participants per condition): arms spread out with arm support (condition "Arms\Support" (AS), Fig. 4(a)), arms spread out without arm support (condition "Arms\No support" (ANS), Fig. 4(b)), arms along the body (condition "No arms" (NA), Fig. 4(c)). In the condition of NA, the participants were asked to place the palm of their hands on their thighs in whichever position they found most comfortable. For these three conditions, participants were using the flight style reported in [25], where torso movements are mapped into pitch and roll input for the drone as illustrated in Fig. 4(e) and (f). The orientation of the pilot's trunk is computed with a gradient descent algorithm implemented in the MCU based on the measurements of the IMU located in the middle of the upper back (see Fig. 3(a)). Such an estimation method was selected for its performance and ability to operate at low sampling rates, which significantly reduces the power consumption [33]. Since the body movement dynamic is low, the gradient descent algorithm was run at a frequency of 20 Hz, which in pilot experiments proved to be a reasonable compromise between tracking accuracy and power consumption. Within these conditions, the static RMS error of the IMU was less than 2°and the dynamic RMS error less than 7°. Moreover, we performed tests at various sampling rates and no differences were identified between them.
For a control experiment, 8 participants controlled the simulated drone with a conventional remote controller (condition "Remote controller" (RC), Fig. 4(d)). Pitch and roll were controlled by moving the joystick up and down, and left and right, respectively.
All participants flew a drone in a simulator developed in Unity3D (Unity Technologies, San Francisco, CA, USA). The simulator physics is based on the eBee, a fixed-wing drone developed by SenseFly (Parrot Group, Paris, France), flying at a constant cruise speed of 12 m/s. This is the nominal speed of drones during imaging and mapping tasks.
Half of the participants had no prior experience with simulated or real aircraft flying operations. After statistical analysis, no correlations were found between the prior experience on flying a simulated or real aircraft and the performance of the participants in any conditions. Also, participant level of expertise in between conditions was statistically the same. The same analysis was done with the number of hours of computer gaming and the same results were found. During the tasks, participants were asked to fly through waypoints represented by small clouds (see Fig. 4(g) and 4(h)). These waypoints were forming a trajectory in the sky and disappeared when they were reached. The experiment was constituted of a training period of 5 minutes followed by two evaluation tasks where participants had to reach 34 waypoints positioned at high altitude ("open environment", Fig. 4(g)) and then 34 waypoints positioned between buildings ("cluttered environment", Fig. 4(h)). The second task aimed to be more stressful and required sharper changes of direction as participants had to avoid buildings. An arrow pointing toward the next waypoint to reach is used to help the participant.
To compute the performance, the distance between the center of the waypoint and the point where the trajectory of the drone crosses a plane drawn perpendicular to the line connecting the previous and next waypoint is recorded for the 34 waypoints [16]. Performance per participant is computed as the root mean square (RMS) of this distance over the 34 waypoints of a task.
At the end of the experiment, participants filled out a questionnaire about comfort and immersiveness. Questions were asked in the form of a Likert scale (an example is given in Fig. 6(a)). To compare performance and questionnaire answers between conditions, a Wilcoxon Rank Sum test was performed and, in order to analyze the variance difference between conditions, an F-test for equal variances was performed. All calculations done in this study, including statistical analysis and correlation, were computed in Matlab (MathWorks, Massachusetts, USA).

A. Flight Control Performance
No significant difference in flight performance was found between the conditions with the exosuit and the condition with the remote controller, for tasks in both the open environment (see Fig. 5(a)) and the cluttered environment (see Fig. 5(c)). These results suggest that the exosuit design and control strategy were comparable to a conventional remote controller. The higher overall RMS error for all conditions in the cluttered environment task with respect to the open environment is suggestive that it is a more difficult task.
However, significant differences were found in both tasks for the three exosuit conditions: in the open environment task, NA condition was significantly worse than the AS condition (p = 0.021), and in the cluttered environment, the NA condition was significantly worse than both AS (p = 0.021) and ANS (p = 0.049) conditions. These results suggest that subjects were more accurate in controlling the drone with the torso motion when the arms were spread out.
Furthermore, the performance variance was significantly larger in the RC condition than in the AS condition in both the open environment task (p = 0.006, Fig. 5(b)) and in the cluttered environment task (p = 0.002, Fig. 5(d)). This suggests that some subjects had more difficulty in piloting the drone with the remote controller than with the exosuit with the arm support. The same observation can be made when comparing the performance variance of the NA condition that is significantly larger than AS condition both in open environment (p = 0.009, Fig. 5(b)) and cluttered environment (p = 0.008, Fig. 5(d)).

B. Questionnaire Results
At the end of the flight sessions, participants had to fill a questionnaire where they rated on a Likert scale (from 1 to 7) how strongly they agreed with a set of statements (an example is shown in Fig. 6(a)). These are displayed on top of each sub figure b to f of Fig. 6. The statements were measuring the subjective feeling of the participant regarding the proposed gesture controls (Fig. 6(b)-(d)) and their physical comfort (Fig. 6(e) and (f)).
Significant differences for the gesture control were found between the condition using the exosuit with the arms spread out (conditions AS and ANS) and the RC. Participants felt better control over the flight trajectory in the condition AS than with the RC (p = 0.009, Fig. 6(b)). Significant differences were also found in the kind of gesture proposed; both AS and ANS conditions have significantly better rating than the RC condition, as shown in Fig. 6(c) (p = 0.002 between AS and RC, and p = 0.001 between ANS and RC) and Fig. 6(d) (p = 0.015 between AS and RC, and p = 0.012 between ANS and RC). In addition, participants would use the flight gestures with the arm spread out more than with the arms along the body (Fig. 6(c), p = 0.034 between AS and NA, and p = 0.018 between ANS and NA).
Regarding the physical comfort, participants rated a significantly higher discomfort with the remote controller than with all the flight conditions using the exosuit (Fig. 6(e), p = 0.009 between AS and RC, p = 0.001 between ANS and RC and p = 0.021 between NA and RC). Participants felt more arm fatigue when flying in the condition ANS than all three other conditions ( Fig. 6(f), p = 0.023 between ANS and AS, p = 0.007 between ANS and NA and p = 0.023 between ANS and RC). However, they felt the same level of arm fatigue between conditions AS than NA and RC.

IV. FLYING WITH REAL DRONES
The FlyJacket has been tested for the teleoperation of a real drone. The flight was performed with a quadcopter (Bebop 2, Parrot Drones, Paris, France) mimicking the flight dynamic of a fixed-wing drone (see details of the control in [16]). The drone can stream real-time video feedback to the goggles of the user. The test simulates a search-and-rescue mission where the user operates a drone to geotag points of interest, for example injured people or dangerous areas. These points of interest will subsequently populate a map that can facilitate the planning of the intervention. In the test, the user was wearing the FlyJacket with the arm support and a smart glove capable to detect predefined finger gestures through capacitive sensors placed on each finger and on the palm (see Fig. 7(a) and (b), and the complementary video). For example, setting a point of interest is triggered by pressing the middle or the ring finger against the thumb depending on the point category. The glove can be also used to send high level commands to the drone, for example automatic takeoff, landing and return home. The trajectory of the drone can be followed on a computer (see Fig. 7(c)). Red and green dots are points of interest set by the user during the flight. Different colors can be used to describe the nature of the recorded point, for example, in the simulated rescue mission, green for injured people to be rescued and red for dangerous areas. The point of interest appears in the center of the field of view of the drone (white cross in Fig. 7(d)). The numbers above the recorded points indicate the estimated distance between the point and the drone in meters. Points of interests can be added and removed from the flight directly during the flight. Fig. 7(e) shows the log of the torso body movements and the corresponding response of the drone orientation for both pitch and roll.

V. DISCUSSION
The presented exosuit, the FlyJacket, addresses the challenges of controlling a fixed-wing drone in a natural and immersive way while being portable and fitting various morphologies. The whole system can easily fit in a backpack and be transported in the field. A simple but efficient arm support prevents arm fatigue without reducing user's performance. Indeed, experimental results demonstrate that participants found the jacket comfortable, natural and intuitive, and were able to easily control a simulated drone with it. The level of discomfort was very low and participants used the free comments section of the questionnaire to report that they only felt dizziness due to the virtual reality goggles, a well-known problem [34], which is not caused by the exosuit.
The flight performance of the exosuit is comparable to a remote controller. However, the performance variation across the participants is significantly smaller when using the exosuit with the arm support in both open and cluttered environments. These results show that all participants could reach a low RMS error using the exosuit with the arm support unlike participants using the remote controller. A possible explanation for the significantly lower performance variance when users pilot with the torso, is that the visual feedback, that is present also while flying with the remote controller, is complemented by the vestibular and somatosensory feedbacks that have a direct correlation with the attitude of the drone [35].
In addition, the use of the exosuit with the arms spread out is preferred by the participants, and in general they find this control more natural than flying with the remote controller. They also have greater feeling of controlling the trajectory. Moreover, participants reported in the questionnaire significantly less physical discomfort using the FlyJacket than the remote controller. This difference is caused by a higher level of dizziness felt when flying using the remote controller. We hypothesize that, as the participant's body stays static when using the remote controller, the sensation of motion sickness due to the virtual reality goggles is intensified (as suggested also in [34]).
When comparing the different flight styles of the exosuit, the RMS error is significantly higher when flying with the arms along the body than when flying with the arm spread out with the arms support. The median of the RMS error is also higher than for all other flight conditions. The error difference is even greater when participants fly in a cluttered, and therefore more stressful, environment in which more precise manoeuvers are required. One possible explanation is that participants had difficulties performing precise maneuvers with their arms along their body, while having their arms spread out would allow them finer control of the rotation of their torso. This could be explained by the fact that small torso rotations correspond to large displacements of the arm extremities, which could improve proprioception [36]. In addition, regarding the questionnaire results, both flight styles with the arms spread out are preferred by participants.
There were no significant differences in participants' performance and feeling when having the arm support or not. Moreover, performance is not prevented by the support and it does not induce any discomfort. The only difference is that the feeling of arm fatigue is significantly higher when flying with the arms spread out and no support than for all other flight conditions. The same low level of fatigue was perceived by the participants flying with the arm support as the one flying with the arms along the body.
Finally, the FlyJacket has been demonstrated for the teleoperation of a real drone. The user performed a survey flight to locate points of interest on the ground. As the exosuit leaves the hands free, the user can easily grasp objects such as the head mounted display when setting up the device. Finally, the free hands allow to exploit hand-held devices in order to perform complementary tasks such as geotagging points of interests or sending high level commands to the drone.