B:Ionic Glove: A Soft Smart Wearable Sensory Feedback Device for Upper Limb Robotic Prostheses

Upper limb robotic prosthetic devices currently lack adequate sensory feedback, contributing to a high rejection rate. Incorporating affective sensory feedback into these devices reduces phantom limb pain and increases control and acceptance. To address the lack of sensory feedback we present the B:Ionic glove, wearable over a robotic hand which contains sensing, computation and actuation on board. It uses shape memory alloy (SMA) actuators integrated into an armband to gently squeeze the user's arm when pressure is sensed in novel electro-fluidic fingertip sensors and decoded through soft matter logic. We found that a circular electro-fluidic sensor cavity generated the most sensitive fingertip sensor and considered a computational configuration to convey different information from robot to user. A user study was conducted to characterise the tactile interaction capabilities of the device. No significant difference was found between the skin sensitivity threshold of participants’ lower and upper arm. They found it easier to distinguish stimulation locations than strengths. Finally, we demonstrate a proof-of-concept of the complete device, illustrating how it could be used to grip an object, solely from the affective tactile feedback provided by the B:Ionic glove. The B:Ionic glove is a step towards the integration of natural, soft sensory feedback into robotic prosthetic devices.

Finally, we demonstrate a proof-of-concept of the complete device, illustrating how it could be used to grip an object, solely from the affective tactile feedback provided by the B:Ionic glove. The B:Ionic glove is a step towards the integration of natural, soft sensory feedback into robotic prosthetic devices.
Index Terms-Haptics and haptic interfaces, prosthetics and exoskeletons, soft robot applications, soft sensors and actuators, wearable robotics.

I. INTRODUCTION
T HE loss of a limb after amputation inevitably changes a person's lifestyle. Both motor and sensory functions are lost, significantly limiting their ability to perform daily activities and affecting their quality of life [1]. For an upper limb amputee, motor functions can be partially re-established by means of a myoelectric or body-powered prosthesis. However, restoring natural sensory functions remains challenging. The lack of sufficient sensing is a contributing factor to 1 in 5 upper limb amputees opting to not wear an upper limb prosthesis. Of those that do, approximately a third (depending on the type of device) end up rejecting it [2].
Amputation of a limb disrupts the sensory-motor control loop of the amputee. This closed feedback loop consists of an efferent pathway from the brain to the limb for movement (motor control) and an afferent pathway that sends sensory signals from the limb to the brain (sensory control) [3]. Robotic prostheses focus on reinstating the motor part of the missing limb, with less consideration of the sensory feedback that is a crucial part of the sensory-motor loop. Furthermore, phantom limb pain is experienced in 60-80% of amputees [4]. Closing the sensory-motor loop can help reduce phantom limb pain, in addition to improving control of the prosthesis, and therefore increasing acceptance [5].
To address this challenge, efforts have been made to bridge the robot-body gap and deliver natural two-way communication between the amputees and their prostheses [6]. A common strategy is to control motion of the robotic prosthesis through muscle activity in the residual limb sensed by skin surface electrodes. These signals can be coupled to microelectrode arrays implanted into sensory nerves in the residual limb to provide sensory feedback [7]. This approach is invasive and requires surgery to embed electronic components in the body, which could lead to complications.
A non-invasive approach is to provide sensory feedback from the prosthesis to the surface of the residual limb, e.g. through stimulation of the skin [6]. The skin is a suitable target to act as a communication channel since it is the largest sensory organ of the body and contains a range of cutaneous mechanoreceptors. These are widely distributed and are capable of detecting a multitude of stimuli including touch, vibrations, stretch, temperature, texture and pain [8].
Many haptic interfaces currently focus on vibrotactile feedback. In upper limb amputees this increases their ability to grasp objects with their prosthetic arm without the need to rely on their vision [9], [10]. However, vibration receptors have a large receptive field, which severely limits spatial resolution and activity [8]. Furthermore, vibrotactile feedback is disruptive to user concentration and may be less suitable for long-term stimulation [11]. Research has therefore turned to other modes of cutaneous stimulation that can simulate more natural and localised sensations.
Receptors that respond to pressure have a much smaller receptive field than vibration receptors and can distinguish between closer stimulation sites on the skin [12]. Antfolk et al. [13] used pressure as a means of tactile feedback, correlating to the pressure experienced at the fingertips of the prosthesis. Huaroto et al. [14] also used pressure, with the feedback system integrated into a prosthetic liner that sits between the socket and the skin.
Another modality of skin innervation that has seen recent interest is skin-stretching. This can be used to provide proprioceptive information about the opening/closing of the prosthetic hand as seen in the Rice Haptic Rocker [15], [16]. The use of shape memory alloy (SMA) actuators for stretching the skin has become more common over recent years. HapticClench, a device by Gupta et al. [17], investigated squeezing pressure feedback that provided tangential and shear forces with the use of SMAs around the wrist and finger. The Tickler [18] is also worn around the wrist but utilises SMAs to move parallel bars laterally across the skin to generate natural and pleasant sensations. A device by Haynes et al. [11] is adhered onto the skin where the sensations generated by the SMAs are found to be less intrusive compared to vibrotactile stimulations.
In this letter, we introduce the wearable B:Ionic glove ( Fig. 1  and 2), a prosthetic sensor system capable of providing mechanotactile stimulation on the user's arm relative to the pressure experienced at the fingertips of their upper limb prosthetic device. It consists of pressure pads containing conducting fluid located at the fingertips of the glove which can be easily attached to a prosthetic device. When pressure is applied, this fluid travels through a network of silicone channels, connecting pairs of electrodes and closing electrical circuits. These electrical circuits initiate contraction of corresponding SMA actuators located on an armband placed on the user's residual limb that gently squeezes their arm.
This letter presents individual characterisation of the three components that make up the sensory feedback device, followed by a proof-of-concept demonstration illustrating how a user may use the system to grasp an object without the need for visual feedback.

II. SYSTEM DESIGN
The B:Ionic glove ( Fig. 1) consists of three components: i) a glove worn on the prosthetic hand with a soft sensor pad underneath each fingertip connected to silicone channels and containing a conductive liquid, ii) an electro-fluidic controller (Soft Matter Computer), and iii) a tactile armband. When the pad is pressed, the volume change forces the conductive liquid through a network of silicone channels. This fluid then bridges the gap between pairs of electrodes along the channel, closing the electrical circuit. In its simplest form, each pad is connected to one coiled SMA actuator on the armband. Once the circuit has been closed, the corresponding SMA on the armband contracts as the current drawn from the battery heats the SMA actuator, generating skin-stretching and squeezing sensations on the skin.

A. Pressure Pad Tactile Sensors
The pads are fabricated using silicone elastomer (Dragonskin 10, Smooth-On). They are cast in two parts using 3D printed moulds. After curing, the two silicone parts are joined using Sil-Poxy adhesive (Smooth-On) to form an enclosed cavity as illustrated in Fig. 3. The pads have an outlet channel that wraps around the finger so that the silicone tube inserted into this channel runs along the dorsal side of the hand to reduce possible interference. This tube acts as the physical connection between  the pads and the Soft Matter Computer and carries the fluid that is used for communicating touch.
The relation between the force applied to the fluidic pad and the displacement of fluid in the channels was characterised as follows. A single pad with an outlet channel was affixed to a linear stage. The pad was filled with a coloured fluid and the initial position of the meniscus in the channel was marked. As the linear stage moved, a probe pressed down onto the pad. The force applied on the pad was detected by a force transducer placed underneath it. Simultaneously, the displacement of the fluid meniscus in the channel was recorded on camera. Pads with three different cross sections and the same internal volume were tested: rectangle, square and circle. A graph of the relationship between force and displacement of the fluid is shown in Fig. 4 with the mean and standard deviation of three trials for each shape.
The circular pressure pad proved most sensitive, showing the least amount of force for the same fluid displacement. This may be due to the corners of the square and rectangular designs increasing the tension across the surface of the pad as it is pushed, making it harder to displace the liquid compared with the circular design. A displacement of 25 mm required a force of 2.5 N (σ = 0.7 N), 2.1 N (σ = 0.4 N), and 1.4 N (σ = 0.9 N) for the rectangle, square and circle geometries respectively. Positional deviation of the probe across trials and complex local interactions at the fluid-wall interface might influence the variance. The circular pad design was chosen for the following experiments and in the glove prototype.

B. Electro-Fluidic Control
The control unit is based on the Soft Matter Computer (SMC) developed by Garrad et al. [19]. For this device, the SMC was made out of silicone (PDMS, Sylgard 184, Dow Corning) cast into a 3D printed mould. Salt water was used as the ionic-conductive fluid. It has the benefit of being low-cost and non-toxic in case of leakage. The SMC contains five channels for silicone tubes to slot in to. Each channel has two holes separated by 10 mm into which gold plated copper electrodes can be inserted and then sealed with silicone adhesive (Sil-Poxy, Smooth-On). Wires are soldered to each electrode and connected to a DC battery which is turned into an AC current via an H-bridge circuit. AC is required to prevent electrolysis of the salt water.
To demonstrate the potential SMC control of the device, we developed a system that would allow for more complex tactile information processing and feedback (Fig. 5). Two pressure pads, A and B (representing two fingers), were set up with an OR gate followed by an AND gate along the tubing. The channels were filled with saturated salt water (40 g NaCl per 100 ml water) to the level such that pressing either of the pads activates the OR gate, and pressing both pads activates the AND gate. The SMA actuators were substituted for LEDs to visualise the moment the fluid made an electrical connection, which was recorded on camera. A gap of 30 mm between the OR and AND gates ensured a clear separation between the two different signals. The working system is demonstrated in the Supplementary Video.

C. Tactile Armband
To provide mechanotactile stimulation to the user's upper limb, we designed a wearable haptic interface (Fig. 6). The design of the tactile armband was based on the Squeeze armband presented in [20], but further developed and optimised for this study. The armband consists of five re-entrant hexagon auxetic units arranged end-to-end, with the addition of hinges along the beams allowing for strong contraction of each unit. It was 3D printed (Wanhao Duplicator i3) with flexible filament (TPU, RigidInk). Coiled SMA wires (BioMetal Helix, BMX series 150) were connected across the centre of the units and secured with glue. When in its relaxed state, the auxetic units are open and in a square shape. When the SMAs are activated, they contract, squeezing the skin in an axial direction. Small PLA printed circles were also adhered to the underside of the armband to increase the sensations felt at these contact points on the skin. Due to the auxetic nature of the armband, contracting one auxetic unit will also result in a shortening or contraction of the armband,  generating additional circumferential and radial forces on the user's arm (Fig. 1).

III. USER STUDY
We investigated subjective responses to the sensations generated by the armband on 20 healthy and non-amputee volunteers (8 female; 12 male; age range 20-50yrs). The armband was tested in isolation to ensure that the responses provided by the participant were directly related to the armband only. The armband was placed on the participant's arm and activated by an external power supply, with a minimum of 5 s between each stimulation to allow the SMA actuators to return to their initial state. Participants were visually isolated to prevent them from seeing any actuation which may interfere with their responses. The user study comprised two parts: determining sensitivity threshold and identifying pressure and position.

A. Sensitivity Threshold
The first part of the user study aimed to determine the sensitivity threshold of the participants' skin; the lowest value of power supplied to the SMA actuator that creates a noticeable sensation on the participant's skin. Two sites for the armband were used: the lower-part of the upper arm and the upper-part of the lower arm, approximately 3 cm from the elbow joint either side (Fig. 7). We used Psychophysics Toolbox Version 3 (PTB-3) on Matlab (2019b) to run the QUEST adaptive algorithm as described by [21]. One SMA of the armband was stimulated (central location) with an initial random estimated threshold of μ = 0.1386 W (σ = 0.03465 W), correlating to 200 ± 100 mA through 3.465Ω. Depending on the response of the participants, the algorithm would provide a greater or lesser current for

B. Strength and Location Mapping
The second part of the user study aimed to determine the ability of the participants to distinguish between different strengths and different locations of activation. First, one SMA (central location) was activated randomly at three different powers relative to the participant's threshold value: 1.1, 1.5, and 1.9 times the threshold value. As P = I 2 R (P = Power; I = Current; R = Resistance), where R is assumed to be constant, this correlates to 1.05, 1.22 and 1.38 times the current at threshold. The participants were asked to rate the strength of the sensations on a scale of 0-3 (0 = did not feel, 3 = strongest). Secondly, all SMAs were activated individually in a random order at 1.5 times the power of the participant's threshold value. The participants were asked which SMA was activated.
Participants found it difficult to distinguish between different powers of actuation of the armband (Fig. 8, left), with only 66% correct responses. The three levels of power used for differentiation were chosen to ensure a range within the capabilities of the SMA wires (150-400 mA) which was tailored to their skin sensitivity. A lower power would result in the SMA contracting more slowly and with a lower contraction than a higher power. Higher powers also require a longer period for cooling and participants noted that it was sometimes the relaxation that they felt as opposed to the initial squeezing. In general, people with a lower skin threshold found it easier to differentiate the lowest strength from the highest two, whereas people with a higher threshold found it easiest to distinguish the highest strength from the lowest two. This may be because when the sensitivity is high, the highest strength is close to the maximum capability of the SMA and some people recorded they could feel a slight heat from the SMA wires.
In comparison, participants found it easier to detect the different locations of tactile stimulation with 81% correct responses. However, where users incorrectly guessed a position, they almost exclusively selected the position next to the correct one The user wears the armband and controls a two fingered gripper (outlined with a red dashed line) bearing pressure pads to grasp a Rubik's cube using the sensory feedback encoded by the control unit (OR/AND gates).
( Fig. 8, right). Some participants found the outer two locations of the armband (i.e. correlating to the thumb and little finger) the hardest to distinguish. This may be due to the thicker velcro strap at the end of the armbands making it stiffer. Most participants could distinguish between three particular regions of stimulations: region containing location 1 and 2, region containing location 3, and region containing location 4 and 5. As location 3 was used throughout the previous part of the user study, participants may have become familiarised with the location of this stimulation.

IV. PROOF-OF-CONCEPT
To demonstrate how the device could be used in a real-world scenario, we set up a two fingered robotic gripper controlled by a user wearing the armband (Fig. 9). A fluidic sensor pad was attached to each finger of the gripper and connected using the OR/AND gate logic as described above (Fig. 5). A screen was placed between the participant and the robotic gripper to hinder visual cues and force the participant to rely solely on the tactile feedback provided by the B:Ionic glove. The objective of the task was to grip a Rubik's cube placed between the fingers of the grippers with sufficient force to allow it to be picked up. This test highlights how the glove could offer a sense of touch for upper-limb amputees who currently can only rely on their vision to grasp objects [16]. In addition, the glove could assist them in better handling of delicate objects. Both gripper fingers were controlled independently by separate servo motors connected to switches with 3 modes: closing, stop, and opening. The user attempted to close the gripper until both pressure pads were in contact with the object.
If only one of the pads is in touch with the object, only the SMA connected to the OR gate contracts. When both pads contact the object with sufficient force, the SMA connected to the AND gate also contracts, allowing the user to identify when the object is safely gripped. The OR/AND gate logic operates as a discretised version of pressure sensing. We have successfully demonstrated grasping of a Rubik's cube using the B:Ionic glove sensor system using only tactile feedback from contraction of the SMA actuators (Supplementary Video). We also showcase how a possible control architecture could be achieved with simple OR/AND gate logic. Although only one user participated in this proof-of-concept study and only grasping of a Rubik's cube was tested, we aim to expand this to a full-scale user study in future work to investigate the effectiveness of the glove and the tactile response of the system on various objects. The successful demonstration of working OR/AND gate logic, though simple in this first demonstration, opens up the possibility to perform complex computations in more elaborate tasks [19].

V. DISCUSSION
This letter presents the B:Ionic glove, a wearable tactile feedback device for use with upper limb prosthetic devices. Experiments were conducted to test the ability of the device to bridge the sensory gap between a prosthesis and the skin. The various components were characterised separately and a prototype of the whole device is shown in Fig 2. The geometry of the pressure pad affects the sensitivity of the device (Fig. 4), however sensitivity can also be adjusted by the level of fluid in the channels, where more fluid will require less force to close the electrical circuit. This can be fine-tuned by the user for specific tasks and will be addressed in future work.
The SMC controller has scope for further control and computation, such as integral memory. For example, fluid could move from one sensor channel (e.g. A) to another (e.g. B) after a grasp, self-adjusting sensitivity ready for the next grasp.
Further factors that could affect the control, especially in terms of the time delay in the system, include volume and viscosity of conductive fluid, diameter of the channels, space between the gates, and space between pairs of electrodes.
The lower and upper arm did not have significantly different sensitivity thresholds. The data presented in [22] shows that the two-point discrimination distance is greater on the upper arm than the lower arm. This suggests that there are more sensory nerve endings present on the forearm and therefore we would expect the sensory threshold for the forearm to be lower. More trials of experiments are needed to study this. Forces exerted on the user's skin by the armband were not measured directly, however previous work on a tactile device actuated by the same coiled SMA actuators shows a force of approximately 1.25 N when actuated at 2.5 V for 2s [11].
The participants found the location differentiation test to be intuitive. The normal pressure distributed across the arm generated by the auxetic nature of the armband did not interfere with the participants' ability to distinguish between actuation sites. The two-point discrimination threshold as stated by [22] for the forearm is approximately 38 mm. The distance between the SMA actuators on the B:Ionic armband is 32 mm, so increasing this distance may also improve user's ability to distinguish between different sites of stimulation. We are currently extending the preliminary user studies in this work to optimise the number, placement, and strength of the wristband actuators and to widen the scope of the user group in order to increase the repeatability of the results.
Efforts were made to ensure all SMA actuators were of the same length but variation in resistance from 3.31-3.70 Ω was found. Lower resistance instilled a stronger SMA response. Further work would need to characterise this.
The armband was not moved after measuring the participant's sensitivity threshold, as a slight change in location on the skin, or tightness of the strap, could potentially change sensitivity. Care was taken to ensure the armband was placed in the same position for all participants with the middle actuator in line with the participant's middle finger when resting their arm on the table in front of them. We know from literature that skin sensitivity varies not only from proximal to distal areas of the arm, but also circumferentially around the arm with more sensitive skin on the medial and posterior forearm for women and men respectively [23]. We allowed enough time between stimulations for the SMAs to return to their relaxed state. However, repeated stimulation of the same area of skin could lead to saturation of the mechanoreceptors and therefore a lower sensitivity.
In general, people tended to find the device pleasant and stated the sensations as "tingling", "twitching", "something crawling", or "muscle activity".
To continue the development of the B:Ionic glove, we are currently performing further user evaluations. While the evaluation in this letter focused on assessing the performance of the individual glove components, these further experiments will measure the performance of the entire system on a range of every day tasks including grasping a range of commonly used objects. We are also planning a series of experiments involving users of prosthetic limbs. This will allow us to measure task performance in a realistic scenario, while also gathering qualitative feedback about ease of use and device comfort. For the B:Ionic glove to operate in the real world, it must be capable of operating untethered, with power consumption low enough to enable long-term use. The SMA actuators used in the wristband require an operating voltage of 1.5-3 V, meaning they can be powered by a small lithium polymer battery. By using an electro-fluidic control scheme, we eliminate the need to power pressure sensors or a micro-controller. While the electro-fluidic control scheme currently requires additional electronics to generate an AC signal, we are currently investigating alternative conductive fluids to overcome this limitation.

VI. CONCLUSION
This letter presents the proof-of-concept of the B:Ionic glove which has the potential to be used as a sensory feedback device with upper limb robotic prostheses. The device is completely soft and wearable with on-board computation. We have shown that the device can be used to relay different strengths and locations of pressure from prosthetic fingertips to the user's skin, with scope for more complex computation. In future work, we will test the device on upper limb amputees to assess the device in real applications. With this device, they may be able to grasp objects more naturally and intuitively without relying solely on visual feedback. This could reduce phantom limb pain and increase embodiment, consequently increasing acceptance of the prosthetic device and reducing the current high rejection rates.
Underlying data are openly available from the University of Bristol data repository, data.bris, at https://doi.org/10.5523/ bris.3izox36nrlotg2papldp8lyfjc. Work was undertaken under the University of Bristol ethics number 108 884 (approved September 22, 2020).