In-Situ Measuring sEMG and Muscle Shape Change With a Flexible and Stretchable Hybrid Sensor for Hand Gesture Recognition

The accurate recognition of hand motion intentions is an essential prerequisite for efficient human-machine interaction (HMI) systems such as multifunctional prostheses and rehabilitation robots. Surface electromyography (sEMG) signals and muscle shape change (MSC) signals which are usually detected with different types of sensors have been used for human hand motion intention recognition. However, using different sensors to measure sEMG and MSC respectively, it would be inconvenient and add deploying difficulty for human-machine interaction systems. In this study, a novel flexible and stretchable sensor was fabricated with a nano gold conductive material, which could simultaneously sense both sEMG and MSC signals. Accordingly, a wireless signal acquisition device was developed to record both sEMG and MSC signals with the fabricated hybrid sensors. The performance of the proposed in-situ dual-mode signal measurement (IDSM) system was evaluated by the recording signal quality and the accuracy of hand gesture recognition. The results demonstrated that by using two pairs of the hybrid sensors, the proposed IDSM system could obtain two-channel sEMG at a noise level of about $0.89~\mu $ Vrms and four-channel MSC with a resolution of about $0.1~\Omega $ . For a recognition task of 11 classes of hand gestures, the results showed that only with two pairs of the hybrid sensors, the average accuracy over all the subjects was 95.6 ± 2.9%, which was about 7% higher than that with two-channel sEMG and six-channel accelerometer signals. These results suggest that the proposed IDSM method would be an efficient way to simplify the human-machine interaction system with fewer sensors for high recognition accuracy of hand motions.

conductive material, which could simultaneously sense both sEMG and MSC signals. Accordingly, a wireless signal acquisition device was developed to record both sEMG and MSC signals with the fabricated hybrid sensors. The performance of the proposed in-situ dual-mode signal measurement (IDSM) system was evaluated by the recording signal quality and the accuracy of hand gesture recognition. The results demonstrated that by using two pairs of the hybrid sensors, the proposed IDSM system could obtain two-channel sEMG at a noise level of about 0.89 μVrms and four-channel MSC with a resolution of about 0.1 . For a recognition task of 11 classes of hand gestures, the results showed that only with two pairs of the hybrid sensors, the average accuracy over all the subjects was 95.6 ± 2.9%, which was about 7% higher than that with two-channel sEMG and six-channel accelerometer signals. These results suggest that the proposed IDSM method would be an efficient way to simplify the human-machine interaction system with fewer sensors for high recognition accuracy of hand motions.

I. INTRODUCTION
H AND gesture is a commonly used means to operate service and rehabilitation robots. The accurate recognition of hand gestures is essential to realize an efficient human-machine interaction for users who interact with those robots, but it is still a challenge. Different signals related to the arm and hand movements have been used for limb motion recognition [1], [2], [3], [4], [5]. Accordingly, a variety of sensors and systems have been developed to quantify the muscle morphological and electrical changes [6], [7], [8], including stiffness [9], [10], hardness [11], density [12], pressure distribution [13], [14], electromyography (EMG), circumference [15] and electrical impedance [16], and more. Among these signals, sEMG has been studied for decades and has become the most commonly used signal due to its noninvasive and easily-accessible nature. Traditionally, a few electrodes are placed on the limb skin to record sEMG signals for recognition of limb movement intentions. Generally speaking, it is more convenient to record fewer channels of sEMG, but the motion recognition accuracy of the EMG is limited by the number of the electrodes. Using a high-This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ density sEMG sensing system to acquire more channels of signals is an efficient way to promote the accuracy of limb motion recognition. For example, Lizhi Pan et al. used 192 channels of sEMG to improve the robustness [1]; Radmand et al. used high-density force myography (FMG) and achieved a very high classification accuracy for eight movements [14]. However, too many sensors would make the system very complex and hard to deploy, limiting its practical applications.
Another way to improve the performance of limb motion recognition is to combine two or more kinds of signals that relate to the limb movements for a limb motion recognition with a relatively simple recording system. Generally, sEMG, force, and inertial signals generated during muscle activities are combined with each other [17], [18], [19], [20], [21], [22], [23], [24], or one of the three signals is combined with other signals, such as electroencephalogram (EEG) [25], [26], [27], [28], [29], [30], to promote the accuracy of hand gesture recognition. Usually, different signals are collected by corresponding types of front-end sensors (For example, electrodes are used for collecting EMG, strain sensor for force, and IMU (Inertial Measurement Unit) for acceleration). However, if the different types of sensors are placed at different arm sites and multiple acquisition devices are required, the whole system would be complicated in structure and cumbersome in size. Some systems achieve their goals through miniaturized circuits. For example, the Trigno sensor integrates electrodes and IMU sensors on the circuit board, can collect EMG and IM (Inertial Measurement) signals at the same time, and is widely used in the field of HMI. However, due to the sensor with such a separate design structure, it is difficult to make the acquisition system small and light enough.
Fortunately, modern materials and fabricating techniques have made it feasible to integrate multi-signal sensors. Several previous studies reported different types of hybrid signal sensors that can record multiple signals simultaneously [31], [32], [33], [34], [35]. And the portable and integrated data acquisition systems have demonstrated their advantages in various applications. For example, an accelerometer and a microphone were used to capture the mechanomyogram (MMG) signal in one sensor [34], and a hybrid sensor with a piezo-electric and a piezo-resistive was proposed [35]. While these previous efforts make important progress toward practical applications of hybrid sensors, these rigid sensors still are limited in their practicability. We know that the human body surface is soft, curved, and changeable when muscle contracting. These factors would restrict the practical use of rigid signal sensors. In an effort to overcome the limitation of the conventional rigid sensors, flexible and stretchable sensors have been developed to adapt to arbitrary and dynamically changing body surfaces, which are soft and conformal to the human body [36], [37]. Among them, strain sensors (measure the change in dimension of a body under an external load) are a very important class, which are often used to collect human signals [50], [51], [52]. Researchers around the world have successfully made various soft strain sensors for wearable physiological or motion monitoring [53], while achieving integrated multimodal sensing with simple structures remains challenging, especially for gesture recognition. Previously, we have developed different types of flexible and stretchable sensors to measure the bioelectrical signals [39] and MSC for monitoring limb movement [38], respectively. Besides sEMG signals, our previous study shows that as a morphological signal to monitor the movements of the limb, the MSC should be an additional signal in movement identifications for human-machine interaction.
In this study, we fabricated a hybrid sensor using a new flexible and stretchable conductive material, which could sense both sEMG and MSC signals simultaneously. Compared to the widely used sensor Trigno, the newly proposed sEMG-MSC hybrid sensor (sMHS) is flexible, stretchable, less complicated in structure, and light in weight. Accordingly, an IDSM system was developed to collect sEMG and MSC signals with the sMHS. The performance of the proposed sMHS and IDSM system was first evaluated by examining the elongationresistance characteristics of the sensor and the quality of recording signals. And then the accuracy of recognizing different hand gestures was investigated based on the sEMG and MSC signal recordings with the sensors and IDSM system.

A. Sensor Fabrication
A new flexible and stretchable material was used to build the sMHS sensor [39]. As shown in Fig. 1(b), the top layer of the material is a nanogold film, and the bottom layer is polydimethylsiloxane (PDMS). Fig. 2 (a) shows the process of packaging the stripes, which is fast and simple. Firstly, the material was cut into strips of 6 cm by 6 mm. Then, a piece of conductive copper foil tape of 3 cm by 6 mm was attached to the flexible strip. On the conductive copper foil tape, a soldering point was built to connect the wire from the data acquisition system.

B. Working Principle of the Sensor
Because the sensor's material is stretchable, the conductivity of the sensor changes with its surface area. When the sensor is placed on the arm, the shape change of the muscles causes the sensor's shape to change along its length ( Fig. 2 (b)), and the sensor's resistance changes accordingly. Since the length of the sensor is far greater than the width, its width can be viewed as a constant when it is relatively slightly stretched. The sensor can then be viewed as a variable resistor whose resistance varies with its length. Therefore, the shape change of the muscle can be detected through monitoring the change of resistance. By applying a current (AC or DC) to the sensor through the leads and measuring the voltage (Fig. 2 (c)), resistance can be easily calculated using Ohm's Law. Following this, a data acquisition system can be developed. Besides, the top layer is a conductive gold film and can be used as an electrode for detecting biopotential signals. Therefore, the sensor can be used to detect MSC and sEMG signals simultaneously.

C. Impedance Model of the Sensor-Body
The impedance of the skin-electrode can be modeled as a resistance-capacitance (RC) model, which can be found in [40]. The impedance between a pair of electrodes includes two parts: the impedance between the electrodes and the skin,  and the impedance of the body. For convenience, these two parts of impedance can be regarded as one. When a pair of hybrid electrodes are placed on the forearm, it can be modeled as Fig. 2(d). R sen1 and R sen2 represent the end-toend resistance of the two sensors, which is approximately 100 . Z AB , Z AC , Z C D , and Z D B represent the resistances between two of the four points of A, B, C, and D in the arm, respectively. According to [40], the resistance of these four impedances is about two to three K in the frequency of 32 kHz, which is much greater than the end-to-end resistance of the sensor (about 100 ). The impedance between A and B: For Then, The impedance between A and B: Similarly, the impedance between C and D: Finally, the model can be simplified as shown in Fig. 2 (e). This means that the resistance of each sensor can be measured separately without mutual interference when it is placed on the arm.

III. IDSM ACQUISITION SYSTEM
The sMHS sensor needs an IDSM data acquisition system adopting a technology named frequency division multiplexing (FDM) to obtain the sEMG and MSC signals. In this section, the structure of the IDSM system and the principle of FDM technology are introduced.

A. Data Acquisition System
The structural diagrams of the IDSM system for the sMHS sensor are shown in Fig. 3(a). This system can acquire four channels of MSC signals and two channels of sEMG signals simultaneously with two pairs of electrodes. It has four parts, including an analog-front-end module, a central control module that has a WIFI transceiver unit, a power supply module, and a graphical user interface (GUI) program running on a computer.
1) Analog-Front-End Module: This module is responsible for acquiring the sEMG signal and MSC signal. Its core consists of four analog-front-end ADS1292R chips (Texas Instruments, USA), which include bioelectrical signal (ECG, EMG, and EEG) amplifiers, a respiration measuring unit, two 24-bit delta-sigma analog-to-digital converters (ADC), and a right-leg drive unit. The acquisition of the sEMG signal is as follows: firstly, all the right-leg drive units of the four ADS1292R chips are connected by a daisy chain and output to the rightleg drive electrode; then, the sEMG signals detected by the sensors are filtered, amplified, and digitalized by ADS1292R. The acquisition of the MSC signal is as follows: the modulator of ADS1292R outputs a 32 kHz excitation signal which is applied to the sensors through an RC net and produces a voltage that contains the MSC signal of the sensor; then the voltage is amplified, demodulated, and finally digitalized. All these are performed by ADS1292R.
2) Main Control Module: This module is a microcontroller unit (MCU) CC3200 (Texas Instrument, USA) which has a 2.4 GHz WIFI transceiver. The MCU controls the four ADS1292R chips through the SPI interface, reading the digitalized sEMG and MSC signals and sending them to the PC through WIFI.
3) Power Supply Module: This module includes a charging circuit for the battery and low dropout regulators for the whole system.

4) Graphical User Interface (GUI) Program:
This program connects the low-level hardware through WIFI, controls the MCU CC3200 to acquire data, and then displays and stores the data on the PC.

B. Frequency Division Multiplexing (FDM) Measuring Technology
Frequency division multiplexing is a widely used technology in the modern communication and measurement field [41]. It divides the total bandwidth into a series of non-overlapping sub-bands, each of which can be used to carry a different signal. sEMG signal and MSC signal are both below 2 kHz, and part of their spectra overlap ( Fig. 3 (b), left). To measure them simultaneously, one type of signal must be transferred to another band by the FDM technology. When a high-frequency current signal, such as the 32 kHz sinusoidal signal in this study, is applied to the sensor, it will be modulated by the resistance of the sensor that works as a multiplier; finally, a voltage v o carrying the resistance signal will be produced, which is a mixed signal with a central frequency of 32 kHz.
Thus, the MSC signal is transferred to the band of 32 kHz. The spectrum allocation of the mixed signal is shown on the right of Fig. 3 (b).
To separate the sEMG signal and the MSC signal in the mixed signal, a low-pass filter, and a high-pass filter are utilized. When the mixed signal passes through the low pass filter, the sEMG signal is extracted. When the mixed signal passes through the high pass filter, the modulated MSC signal is extracted, and the original MSC signal is obtained by demodulation. The demodulator is composed of a multiplier and a low-pass filter. The multiplier uses the carrier to multiply the received signal, and a signal with twice the frequency of the received signal is produced. When this double-frequency signal passes through the low pass filter, it (the right item in formula (6)) is removed, and only the MSC signal is left. This process is shown in Fig. 3 (c).
The FDM is realized through the chip ADS1292R which is a high-integrated analog-front-end chip used to acquire biopotential signals (including EEG, ECG, EMG) and respiration impedance. ADS1292R includes two channels of PGA and ADC, a respiration modulating module, a respiration demodulating module, an RLD (right keg drive) module, and some auxiliary circuits. The respiration modulating module and respiration demodulating module realize the FDM measuring technology in the IDSM system.

IV. EVALUATION OF THE IDSM ACQUISITION SYSTEM
In this section, the performance of the developed IDSM system was tested. The characteristics of the sensor and the quality of the sMHS acquired by the system were studied.

A. MSC Signal Acquisition
1) The Characteristics of the Sensor: The sensor was tested in a tensile machine to find the relationship between the length, the force, and the resistance. The protocol was as follows: firstly, the sensor was pulled (loading) at the speed of 1 mm/s along the direction of the sensors; when it was stretched to 20% of its length, it was relaxed (unloading) at the same speed. This process was repeated 100 times to make the sensor stable, and then the data of 5 circles were recorded ( Fig. 4(a)). Fig. 4(b) shows the relationship between the resistance and the length. The loading and unloading processes do not coincide with each other, and they form a large hysteresis curve. This means that the sensor is not very precise. The relationship between the force and the length is shown in Fig. 5(c), which has better linearity and less hysteresis, compared with the relationship between the resistance and the length.
2) MSC Signal: The measurement of the MSC signal is conducted by measuring the resistance of the sensor (with a sampling rate of 1K Hz). To validate the performance, a standard resistance box (KREVOR, China) was used as a reference. Because the sensors' resistance is about 100 , this system focuses on the range of resistance less than 2000 . Fig. 4 (d)/(e)/(f) show the results of the system measuring the resistance box, where a quadratic function was used to describe the relationship between the voltage obtained on the sensor and the resistance.
v (unit: mV) is the sensor's voltage measured by the system, r is the resistance (unit: ) of the sensor. Fig. 4(d) shows the error that is within ±1%. Fig. 4(e) is an example of measuring the resistance from 200 to 209 . The left part of the curve shows the resistance increased from 200 to 200.9 with a step of 0.1 , while the right part shows the resistance increased from 200 to 209 with a step of 1 . The system can distinguish the change of 0.1 of the sensor (Fig. 4(e)). Fig. 4(f) is the absolute error curve of Fig. 4(e) relative to the standard resistance box, which has a mean value of 0.7 and a standard deviation of 0.05 . B. sEMG Signal Acquisition 1) Input Reference Voltage Noise: Input reference noise is often used to evaluate the intrinsic noise level of the system, which is a very important performance criterion. The protocol is as follows: the gain of the acquisition system (the PGA) was set to 6; the sampling rate was 500 Hz; the inputs of each channel were short-circuited and connected to the RLD through a resistor of 10 K . Then, the output data of the system were recorded. Fig. 5 (b)/(d) shows that the peak-to-peak value of the input reference voltage noise over 10 seconds was less than 8 uVpp, with the corresponding RMS value of 0.89 uVrms. According to [42], the RMS of the input reference voltage noise should be less than 1 uVrms. This indicates that the input reference voltage noise of this system meets the requirements of sEMG measurement. As a comparison, these data were similar to the data provided by ADS1292R's user manual Fig. 5 (a)/(c)).
2) Bandwidth: The bandwidth of ADS1292R depends on the sampling rate of ADC (for there is a decimation filter in  the chip, the bandwidth is determined by this filter), which is about 1/4 of the sampling rate according to the datasheet of ADS1292R. To test the real bandwidth, we fed a sinusoidal signal of 10 mV with its frequency varying from 10 to 900 Hz (a function waveform generator was used, DG 4062, RIGOL, China). The sampling rate of the sMHS acquisition system was set to 2000 and 4000 samples per second (SPS) respectively. The results are shown in Fig. 6. When the system used the sampling rates of 2000 SPS and 4000 SPS, the 3dB bandwidth was 435 Hz and 650 Hz, respectively. The bandwidth was narrower than 1/4 of the sampling rate. This was caused by the front-end anti-aliasing filter.
3) Signal Noise Ratio (SNR): For comparison purposes, a commercial device, the Trigno system (Delsys, USA), was used as a reference. The experiment was conducted on one male subject (41 years old). The electrodes were placed on the brachial and radial muscles of the forearm, as shown in Fig. 7(a). The IDSM system's sampling rate was set to 2000 SPS, and the Trigno's was set to 1926 SPS. The subject was asked to close his hand with about 50% maximum voluntary contraction (MVC) for about five seconds and then relax for about five seconds. The sEMG signals are shown in Fig. 7(b). The waveforms of our device are very similar to the waveforms of Trigno. Fig. 7(c) is the spectra of the two devices, which are also very similar. SNR, as defined in formula (8), is used as an indicator to further quantify the differences between these two systems. Fig. 7(d) shows the signals of the rest state for 4 seconds. Our device and Trigno almost have the same noise (12 uVpp). Finally, the SNR of our device is 36.95 dB, which is 3.62 dB higher than that of Trigno (33.33 dB). The reason for this difference is that the electrode of Trigno is small and the distance between electrodes is close, while the electrode of our equipment is larger with more inter-electrode distance.
S N R = 20log 10 RM S active RM S rest (8)

V. APPLICATION FOR HAND GESTURE RECOGNITION
To validate the recognition performance of the sMHS sensor and the IDSM system, experiments on a hand gesture recognition task were conducted with our IDSM system and the Trigno system (Delsys, USA). The protocol of the experiment was approved by the Institutional Review Board of Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences. All subjects gave written informed consent and provided permission for the publication of their photographs for scientific and educational purposes. The experimental protocol is as follows.

A. Experimental Protocols
Eleven young students were recruited for this study, including six males and five females, whose average age was 26.63±4.92 years. The setup of the IDSM system was as follows. Before the sensors were attached, the skin of each subject was cleared with alcohol swabs. Four sensors were smeared with conductive gel and attached to the forearm to collect two channels of sEMG and four channels of MSC signals. The sampling frequency of the sMHS acquisition system was set to 2000 SPS, and the PGA to 12. As shown in Fig. 8(a), the sensors are located on the two ends of the brachial and the radial muscles of the forearm. For the Trigno system, two sensor units were used, including two channels of sEMG signals and six channels of IM signals. The sensors were placed on the bulks of the same muscles as the IDSM system (as shown in Fig. 8(a)). The sampling rate of EMG was set to 1926 SPS, and that of IM was 149 SPS.
Data of eleven hand gestures (the RS was included) were collected from eleven subjects, including ten active movements and one rest (RS) state. The active movements were hand close (HC), hand open (HO), wrist pronation (WP), wrist supination (WS), thumb up (TM), wrist extension (WE), wrist flexion (WF), index (ID), victory (VT), and OK (as shown in Fig. 8 (b)). Before the data collection sessions, the subjects were properly instructed about the experimental procedures. Each subject was allowed to perform several pre-experimental trials to get familiar with the experimental protocol. Following these procedures, the subjects performed each gesture based on an animate prompt for 5 seconds, and each gesture was followed by a rest session of five seconds before observing the next gesture. In each trial, the order of gestures was as follows: HC, HO, WP, WS, TM, WE, WF, ID, VT, and OK. Each subject was asked to repeat the process three times.

B. Data Processing and Classification
All the data were processed offline, according to the following four stages, as shown in Fig. 9. 1) Data Preparation: Firstly, the sEMG and IM data of the Trigno system were resampled to 2000. Secondly, to suppress the noise, a five-point moving average filter was applied to the MSC and IM data. For all the sEMG data (including the IDSM system and the Trigno system), a bandpass filter with a passband frequency of 25 Hz to 450 Hz was used to remove the low-frequency artifacts. Thirdly, according to the ten active hand gestures, the filtered data related to the gestures were separated and divided into ten classes, while the rest of the data belonged to the rest state. Fourthly, a series of 100 ms analysis windows with an overlap of 50 ms was utilized to segment the data of all classes.
2) Feature Extraction: For each channel of data, four time-domain features (mean absolute value, mean absolute value slope, wavelength, and zero-crossing) were calculated for each analysis window, and were calculated as reference [47]. Therefore, the features extracted from each channel of data formed a four-dimensional vector. For all signals (IM, MSC and sEMG), four time-domain features were extracted as above. The feature fusion of the MSC and sEMG signals for the IDSM system was achieved by concatenating their feature vectors together. Illustrate with an example, two channels of EMG signal would produce an eight-dimensional feature vector. And four channels of MSC signal would produce a 16-dimensional feature vector, and concatenating their feature vectors together would produce a 24-dimensional feature vector. The feature fusion of IM and sEMG signals for the Trigno system was as for the IDSM system above. Because the hold-on time for each gesture was different, the features of eleven gestures (the RS was included) were balanced by randomly selecting the same number of samples for all gestures.
3) Classification: In this study, the accuracy of an offline classification was regarded as the performance index. The feature vectors were divided into a training set and a test set by 5-fold cross-validation, and then classified by linear discriminant analysis (LDA). Compared with other classifiers, LDA is easier to implement, has less computation, and does not affect the accuracy of limb motion classification significantly [43].

4) Statistical Analysis:
To examine whether the type of signals (MSC, sEMG, IM, and their combinations) have an impact on the accuracy of movement classification, the one-way ANOVA is conducted in terms of mean classification accuracy, using the SPSS Statistical Modeling software (SPSS 22.0 IBM Corp., Chicago, IL). A level of P < 0.05 is selected as the threshold for statistical significance with the null hypothesis that the classification accuracies achieved by different signals (such as 4MSC, 2EMG, 6ACC, 2EMG+4MSC, and so on) are not significantly different from each other. If there is a significant difference in the results of ANOVA, LSD (least significant difference) as the post-hoc analysis (multiple comparison analysis) would be conducted.

C. Results
1) The Waveforms of the sMHS: Fig. 8 (c) shows the waveforms of one channel of sEMG signal (in the top portion) from one pair of electrodes and two channels of MSC signals (in the bottom portion) from the corresponding two electrodes, including ten active gestures and ten rest states from a complete trial. These two sensors are located on the brachioradialis ( Fig. 8(a), the left portion). The waveforms of the MSC signal have mean values and standard deviations of 29.0±6.7 and 142±33.0 , respectively. Unlike the sEMG signal, the MSC is a low-frequency signal. It is highly related to hand movements (gestures), so each active movement can be distinguished from the waveforms. For each active movement, the trends of the MSC amplitudes are different from the sEMG. Some movements have low sEMG amplitudes but high MSC amplitudes. For instance, among the ten active hand gestures, WS, WP, and TM have lower sEMG but higher MSC (of the second channel), compared with other movements. This complementary feature of different signals may promote pattern recognition performance.
Then, repeatability was tested, which is a very important index of sensors. For the sMHS sensor, two factors can affect the repeatability. The first one is the nuance of the same gesture in each trial, and the other is the hysteresis of the sensor. Fig. 8(d) shows one channel of MSC signal, including three trials. The gray area represents the standard deviation, which is within ±5 .
2) Accuracies of Hand Gesture Recognition Using the sMHS: To evaluate the performance of the IDSM system for hand The accuracies of classification for these data are shown in Fig. 10. It can be observed that the classification accuracy increases with the number of signal sources. When all the available sEMG and MSC signals are used, the highest accuracy of 95.6±2.9% is achieved.
Using the same sensor, the IDSM system can help to promote classification performance. For one pair of sensors, one channel of sEMG and two channels of MSC were collected and achieved a classification accuracy of 81.3±8.1% which is 28.1% higher than that achieved from one channel of sEMG (53.2±12.3%). For all four sensors, two channels of sEMG and four channels of MSC were collected and achieved a classification accuracy of 95.6±2.9% which is 16.11% higher than that of two channels of sEMG (79.5±5.5%).
From the same muscle, both sEMG and MSC can obtain useful information for motion recognition, and ANOVA analysis shows that there is no significant difference in recognition performance between the two signals. For example, in Fig. 10(a), one channel of sEMG almost achieved the same accuracy as two channels of MSC (p=0.928, Group 1), and two channels of sEMG almost achieved the same accuracy as four channels of MSC (p=0.408, Group2). It is observed that a combination of two channels of sEMG and four channels of MSC achieved similar performance as using a combination of two channels of sEMG and two channels of MSC. This suggests that a configuration of two channels of sEMG and two channels of MSC might be sufficient for hand gesture recognition.
3) Comparison to Trigno in Hand Gesture Recognition: The data acquired by the Trigno system were processed in the same manner as used with the sMHS. As shown in Fig. 10 (b), the EMG signals of the IDSM system and the Trigno system achieved similar performance, while the classification accuracy of 4MSC (76.7±9.8%) was 10.6% higher than that of 6ACC (66.7±17.4%). The classification accuracy of 2EMG+4MSC (95.6±2.9%) was 7.1% higher than that of 2EMG+6ACC (88.5±7.4%). ANOVA analysis shows that there is a significant difference in classification accuracies between 4MSC and 6ACC (p<0.0001), and between 2EMG+4MSC and 2EMG+6ACC (p<0.0001). These results show that the MSC signal is more efficient than the ACC signal for the task of hand gesture recognition. Fig. 10(c) shows the confusion matrixes of the IDSM system, which has an average classification accuracy of 95.6±2.9% (eleven subjects, eleven classes) and the classification accuracy of each movement is over 91%. Fig. 10(d) is the confusion matrix of the Trigno system, which has an average classification accuracy of 88.5±7.4%. For the Trigno system, some movements related to the individual fingers have relatively low accuracies. For instance, the accuracies of the VT and OK movements are less than 80%. The reason is that the muscles related to these movements are small or deep in the forearm, so the sensor units of the Trigno system can only obtain limited information about these muscles. In addition, the IM signal is related to movement, which is not very effective when the hand remains still. On the contrary, the IDSM system can obtain both electrical and morphological information from the muscles, which is complementary and helps improve classification performance. As a result, the IDSM system achieved a higher classification accuracy than Trigno.

VI. DISCUSSION
The experimental results indicated that under the condition of a certain number of channels, a combination of different types of signal sources could be more effective to improve hand gesture recognition performance, due to their complementary characteristics. The findings of this study showed that the combination of sEMG and MSC signals could promote the accuracy of hand gesture recognition, and achieve better performance than solely using sEMG or the combination of sEMG and IM signals. These experiments also suggest that a combination of different types of signals could be an alternative to a large number of single-type signal recordings for motion recognition analysis.
The hybrid signal sensor and in-situ measurement are promising in obtaining multiple signals with fewer sensors. Because muscles are multi-signal carriers (which carry bioelectrical, morphological, and biomechanical signals), sensors with the capability of acquiring multiple signals are reasonable. Our novel sMHS sensor with the IDSM method is not only able to extract bioelectrical and morphological signals but also increase the number of signal channels by two times without adding sensors. These technologies may help us to achieve a more simple and portable acquisition system.
Being conformal to the human body and comfortable to wear, the flexible and stretchable sensors have the potential to develop practical and wearable human-machine interfaces (HMI). In the hand gesture recognition task, the sMHS sensor was proved effective with an accuracy of 95.6±2.9%, using only four sensors. By manufacturing this sensor into arrays, and combining it with multi-frequency and multi-channel FDM in-situ measuring technology, more channels of different types of signals would be obtained, and the accuracy would be further improved. In view of the characteristics mentioned above, the system with flexible and stretchable sensors would have a smaller size, lighter weight, simpler system, and higher motion intention recognition precision.
In addition to the above advantages, this sensor may be beneficial to improve the stability of gesture recognition due to the MSC signal. The robustness is the main issue for the practical application of sEMG-based gesture recognition. Some factors, such as muscle contraction level change and electrode shift [1], [44], [45], [46], will decrease the accuracy of sEMG-based gesture recognition. Firstly, unlike EMG signal, the MSC signal is stable and strong, thus the slight movement of the electrode will not induce interference to MSC as large as to sEMG. Secondly, due to the continuity of muscle shape change, there is a strong correlation between signals when the electrode displacement is small (relative to the length of the muscle). So the change of MSC signal caused by electrode movement is relatively small. Finally, because the MSC signal is directly related to muscle contraction, when the signal is normalized by the muscle contraction level, the effect of the muscle contraction level change would be minimized.
These are exactly what HMI needs.
Although the sMHS sensor and IDSM system show their advantages, some factors that may affect the performance should be considered for practical use.
(a) Amputee subjects. The muscle anatomy of a residual limb is different from an intact limb. Our previous research shows that the combination of MSC and EMG could achieve higher accuracy in the amputee subject than EMG alone, but the accuracy is still lower than in the normal subject [39]. For better performance of gesture classification, a small-sized sMHS sensor array is needed. Moreover, a socket is typically used to attach the prosthesis to the residual limb. The friction and extrusion from the socket will affect the deployment of the sensor and the acquisition of the MSC signal. Therefore, the prosthetic socket should be specially designed.
(b) Practical factors affecting the measurement of muscle shape. Compared with EMG, MSC signal may be less affected by limb position, fatigue, sweating and other factors. However, some other facts, such as the rapid change in temperature [49] may affect the precision of the signal. So the power of the sensor should be limited, and it is recommended that the current flowing through the sensor be less than 1mA (the corresponding power is 0.1mW). Our previous study on locations also shows that locations could affect the MSC signal [48], and the sensors should be placed in locations with more muscles.
(c) EMG sensor pair distance. The distance between one pair of EMG sensors should be kept at least 5mm [42]. In our experiment, the distance is about 50mm which is much larger than the 10mm inter electrode distance for Trigno. From the result (Fig.7(c)), it seems that the farther the distance, the greater the signal amplitude. Although the amplitude of the signal is somewhat different, it may not have a significant impact on the results of gesture recognition.
(d) Sensor size. As a demonstration, the sensor of this work is quite long. Although the sMHS sensors could collect more signals, they would take up more space around the forearm than typical bipolar EMG pairs. Especially for amputees, the muscle size of the residual limbs can be quite small. Thus, smaller sensors are needed. Our previous studies on MSC signals have shown that small sensors and large sensors have similar results for gesture classification, but the accuracy of small sensors is slightly decreased [48]. Fortunately, it is easy to make this kind of high-density sensor array. This would compensate for the decrease in accuracy.
(e) Strain direction. In this work, the MSC sensor is a unidirectional strain measurement sensor, thus it can only measure the deformation in one direction. When the sensor is placed on multiple muscles, it is important to measure the deformation in multiple directions. The measurement of multiple directions can be realized by making sensors with special shapes, for example, the measurement of two dimensions can be realized by making a cross sensor.
(f) Classifiers. The classification accuracy may be further improved if a better classification method is used. Artificial neural network (ANN), K-means nearest neighbor (KNN) and support vector machine (SVM) were also used to classify the data (2EMG+4MSC), and the accuracies for each classifier were 88.95±3.15%, 94.61±1.59%, 95.58±2.87%, 97.66±1.51% respectively. Because the complexity of SVM is higher than LDA, we chose LDA as the classifier in this study.
The limitations of the developed sensor should also be noted. Firstly, the muscle shape change sensor is not precise for its hysteresis characteristic which is caused by the creep of the material. Although the sensor works well in motion intention recognition, its application might be limited where precise measurements are required. Secondly, the sensor is more fragile compared with traditional ones. The top layer of the nanogold film is very thin and soft which can induce two problems: (a) the nanogold film may lead to the change of impedance or even the failure of the sensor due to the scratch of a sharp object (b) The interface is not stable. The sensor is connected to the data acquisition system through a wire, which is rigid. When receiving a large tensile force and generating a large deformation, the connection between the sensor and the wire can break, making the sensor fail to work. To overcome these limitations, our future work will focus on the encapsulation of the sensor (to make it more robust), and the fabrication of the sMHS sensor array, which has the potential to further promote motion recognition performance.

VII. CONCLUSION
This study shows that the proposed flexible stretchable sMHS hybrid sensor and IDSM acquisition system can simultaneously collect EMG signals and MSC signals, realize the in-situ measurement of the two signals, and can be used for gesture recognition.