Introduction
Monitoring joint angles during dynamic activities of the human body, in real-time, and, ideally, in non-contrived (i.e., non-laboratory-based) settings can benefit several application sectors, including healthcare, sports performance, and robotics [1]. For instance, motion capture can help clinicians personalize the rehabilitation of stroke patients, as well as understand the effect of joint movement on hemophilic arthropathy [2], [3]. It can also help clinicians monitor physical activities of patients with Parkinson's disease [4]. In other cases, motion capture can help calculate injury risk to assist in sports therapy [5], instruct movement-based activities such as dancing [6], and more.
State-of-the-art technologies used for motion capture face several limitations, as detailed in [7]. In brief, gold-standard camera-based motion capture (MoCap) systems are highly accurate but are costly, constrained to a fixed/confined environment, and require line-of-sight [8], [9]. Inertial measurement units (IMUs) are portable/wearable but suffer from integration drift [10], [11], [12], leading to error accumulation with time. Potentiometer-based sensors [13], [14] are bulky and do not conform to the size of the patient, making them uncomfortable to use. Finally, bending (strain-based) sensors [15] restrict natural movement and are limited by the number of cycles of use due to repeated strain.
To overcome these limitations in the state-of-the-art, we have recently reported a new class of wearable loop sensors that capture joint angles using transmit and receive loops placed across the joint [16], [17], [18]. The sensors operate based on Faraday's law of induction where the voltage on the receiving loop changes as the joint flexes, and thus, the loops misalign. These sensors are reliable over time and injury-safe, and do not obstruct natural movement. A summary of our previous work is outlined in Table I, with a focus on monitoring knee joint kinematics. As seen, only proof-of-concept studies have been reported to date where rigid (copper-wire) loops have been tested on cylindrical phantom limbs under static scenarios (i.e., the limb is fixed at a given angle each time a measurement is collected) in a tethered setting.
In this work, we take a major step forward and report dynamic measurements of knee joint flexion, in real-time, on human subjects, using flexible, textile-based loops. To cater to the new requirements, we also introduce: (a) a calibration mechanism that helps extract joint angles from the raw sensor output (i.e., transmission coefficient) and (b) dedicated software built in-house to allow for calibration, real-time recording, and data processing. Without loss of generality, tests are conducted using the two-loop planar sensor configuration of [17], suitable for two-dimensional joint flexion/extension angle measurements. Three different activities, viz. walking, brisk walking, and full flexion/extension, all carried out in place (i.e., the subject is not physically moving from their original location), are utilized to test and validate the sensor on ten (10) healthy adults in a tethered setting (i.e., connected to a network analyzer). To further demonstrate the potential of translating this sensor into a fully wearable version, we also report and validate upon phantoms an untethered system.
The rest of the paper is organized as follows. Section II covers materials and methods. This starts with description of the experimental set-up including sensor design, data collection and recording software, camera set-up and validation, and sensor calibration for the tethered system. This is followed by description of the actual experiment and data analysis methods. Detailed results for the tethered system on human subjects are presented in Section III. Section IV compares the system performance vs. the state of the art, discusses the development of an untethered system, and highlights clinical implications. Section V concludes the paper.
Materials and Methods
A block diagram and a picture of the experimental setup are shown in Fig. 1(a). Two planar loop sensors were placed upon the leg (see Section II-A) and connected to a vector network analyzer (VNA, N5235A) via coaxial cables to record transmission coefficient (|S21|) data. The latter inherently includes flexion angle information [17]. The network analyzer was further connected to a laptop which recorded the |S21| data in real-time using our self-built recording software (see Section II-B). Simultaneously, a light detection and ranging (LiDAR) depth camera was used to capture the flexion angles for two purposes, i.e., (a) to provide gold-standard angles as a reference for comparison, and (b) to calibrate the sensor (see Section II-C). Data recorded from the camera was also captured in real-time using our self-built recording software on the laptop.
(a) Block diagram and (b) experimental set-up including human-worn loop sensor connected to a network analyzer to collect |S21| data for knee flexion, camera collecting gold-standard flexion angles as reference for comparison, and data recording and calibration software on laptop.
The experimental protocol can be divided in two steps and involves: (a) calibration of the sensor (see Section II-D), followed by (b) real-time knee-flexion data collection using both the sensor and the camera for a given activity on a given subject. Details of each component of the experimental set-up are discussed next, followed by the actual experiment and data analysis methods.
A. Sensor Design
Per Section I, the sensor consists of two planar loops placed longitudinally upon the body 8 cm above and below the knee joint with one acting as transmitter and the other as receiver. As detailed in [16], [17], the sensor consists of magnetically coupled transmit and receive loops and operates based on Faraday's law of induction to monitor joint flexion. In brief, as the flexion angle changes, it leads to change in relative position between the loops. In turn, the transmission coefficient between the loops changes. Thus, the change in flexion angle is directly mapped to the change in transmission coefficient (or receiver voltage). Ultimately, monitoring the transmission coefficient values allows for seamless monitoring of joint angles.
Our selection for a planar loop sensor design in this study was based on its tolerance to the anatomical shape and size of the limb, as well as its high angular resolution for flexion-only activities (we purposely omitted rotation monitoring in this first study on human subjects to reduce the complexity and potential sources of error) [17]. Each loop had a radius of 4 cm and was soldered to a 102 pF lumped capacitor to make them resonant at 34 MHz, as per the detailed study in [16] (see Fig. 2). Each of the loops were sewn using LIBERATOR-40 conductive e-thread on a 6 cm ×10 cm piece of cotton fabric [19], [20]. An SMA connector was attached to each loop sensor and kept in place using a 3-D printed fixture and plastic screws. This fixture also provides support for the e-thread. It was then tied to the cotton fabric with non-conductive thread for increased durability. The fabric was finally covered in SYLGARD 184 silicone elastomer to increase structural stability, as shown in Fig. 2(a). That is, the elastomer allows loops to maintain their circular shape and prevents deformation over the course of the experiment without altering the sensor performance. Since the sensor operates in the deep inductive regime, the presence or absence of this elastomer has no impact on the resulting performance.
Loop sensor design: (a) fabricated single loop using e-thread, adapter, capacitor, and elastomer; (b) sensor adhered to subject's limb across the knee joint over a leg brace.
B. Data Collection and Recording Software
As shown in Fig. 1, the VNA was connected to a laptop over Ethernet to facilitate the data collection process via recording software. The VNA was set to sweep at a single frequency of 34 MHz and record the |S21| values while sampling at a rate of 60 points per second with the intermediate frequency (IF) bandwidth set to 3 kHz. The recording software was developed with a web-based frontend and a REST application programming interface (API) backend. The frontend was developed using JavaScript to communicate with the backend. The backend was developed using Python 3.8. Flask was used to create the REST API, pyrealsense2 was used to capture data from the camera, and pyvisa was used to communicate with the VNA and collect |S21| data via the Standard Commands for Programmable Instruments (SCPI) protocol.
C. Camera Setup and Validation
To calibrate the sensor and for gold-standard comparison, a depth sensing camera was used to monitor the flexion angle of the limb during sensor data collection. The camera used was the Intel RealSense 515 LiDAR Camera which is capable of capturing color, depth, and infrared video data at a sampling rate of 30 frames per second where each frame has a resolution of 640 × 480 pixels. During data collection, the camera was placed on a tripod facing the limb, approximately 2 m away from the participant. To calculate the flexion angle, three retroreflective markers were placed on the leg to form the vertices of a triangle, as shown in Fig. 1(b). For each frame, the 3-dimensional (3-D) coordinates of each marker were captured by applying thresholding to the infrared image channel to obtain 2-D coordinates, and transformed to 3-D coordinates using the depth channel. Given these coordinates, the angle of the triangle that corresponds to the flexion angle of the limb (say
\begin{equation*}
\theta \left( {a,b,c} \right) = {\tan ^{ - 1}}\sqrt {\frac{{\left( {\left( {a - b} \right) + c} \right)*\mu \left( {a,b,c} \right)}}{{\left( {a + \left( {b + c} \right)} \right)*\left( {\left( {a - c} \right) + b} \right)}}} \tag{1}
\end{equation*}
\begin{equation*}
\mu \left( {a,b,c} \right) = \left\{ {\begin{array}{ll} {c - \left( {a - b} \right)}, & {{\rm if}{\rm{\;}}b \geq c \geq 0}\\
{b - \left( {a - c} \right)}, &{{\rm if}{\rm{\;}}c > b \geq 0}\\
{{\rm invalid}{\rm{\;}}{\rm triangle}}, &{\rm else} \end{array}} \right. \tag{2}
\end{equation*}
To validate the accuracy of the approach, the flexion angle of a phantom limb (Styrofoam limb as used in [18]) was simultaneously measured by the camera and a goniometer (standard clinical device to measure knee flexion angle). This set-up, pictured in Fig. 3(a), uses the metal goniometer to set the limb to a desired angle. Concurrently, the camera and aforementioned approach were used to capture the corresponding flexion angle. Measurements were taken in increments of 10° statically. The root mean squared error (RMSE) between the angles measured by the camera and the angles measured by the goniometer was 0.322°. The results of this validation process can be seen in Fig. 3(b). This validated the camera to be used for gold-standard comparison purposes and calibration during experiments.
D. Sensor Calibration
Sensor calibration is required to convert |S21| data collected from the sensor to corresponding flexion angles. This enables us to obtain flexion angle as the output from the sensor in real-time, as desired. To do this, the participant needs to just perform full flexion and extension while |S21| data from the sensor and flexion angles from the camera are collected in real-time, simultaneously. One example of such temporally aligned raw data is shown in Fig. 4(a). Note that |S21| and camera data do not overlap due to non-linearity in the characteristic curve (|S21| vs. flexion angle) of the sensor [17]. This results in the calibration data seen in Fig. 4(b). As the camera's sampling rate is half that of the network analyzer, the flexion angle data was always linearly interpolated to have the same sampling rate as the network analyzer. Since the calibration curve should be monotonically increasing [16], th|S21| and angle data were sorted independently of each other after alignment. Let
E. Experiment
Ten (10) adult participants were recruited to participate in this study. Each Participant signed the IRB consent form approved by the Office of Responsible Research Practices of the Ohio State University (Approval No.2017H0472, Date: 08/15/2023). These subjects were selected upon no specific requirements, except that they were physically healthy and physically able to participate.
Consent of the subjects was taken prior to experiments. Referring to Fig. 2, a flexible thigh and shin brace was placed above and below the knee joint on the right leg. The brace was used to limit the movement of the sensors, as well as make them more comfortable for the subject to wear. The sensors were then placed on each subject 8 cm above and below the knee and were fixed in location on the limb using athletic gauze. Next, the SMA cables were connected to the sensors. Finally, the camera markers were placed on the participant, and the other end of each cable was connected to the VNA.
Participants were asked to perform three activities: (a) walking, (b) brisk walking, and (c) full flexion/extension, all in place (i.e., the participant is not moving from his/her original position). Calibration was performed based on Section II-D, i.e., the participant was asked to perform one or more full flexions and extensions that spanned their range of motion. During the walking activity, the participant was asked to walk in place for 60 seconds. During the brisk walking activity, the participant was asked to walk in place briskly for 60 seconds. During the flexion and extension activity, the participant was asked to fully flex and extend their leg five times over the course of 15 seconds to capture the range of motion not captured by the walking activities. All three activities were performed up to five times to enable adequate statistical analysis on each participant. After these activities were performed, the sensors were taken off and put back on the subject. Finally, the participant was asked to perform the calibration process and all three activities again to help ensure repeatability.
F. Experiment
Data processing was done using Python. The knee flexion angles were predicted using the data captured from the sensor during each activity and their corresponding calibrations. These angles were compared to gold standard camera angles using two different metrics: RMSE and Pearson's correlation coefficient (ρ). The RMSE measures the accuracy of the sensors while ρ measures the correlation of the angles obtained via the sensor to the gold standard angles.
Results
Results for each participant and activity are shown in Table II.
Results without participants two and six are also included because these subjects had high levels of RMSE due to sensor fabrication errors and are considered outliers. A graphical summary is provided in Fig. 5. As seen, the correlation coefficient shows very strong agreement between the sensor and gold standard angles across all subjects and activities. We see that the flexion/extension activity achieved the best RMSE. This is expected as the sensors are less accurate at lower flexion angles due to their nonlinear calibration curve (see Fig. 4(b)).
Therefore, since the activity covers the entire range of motion of the knee as opposed to the other activities, the RMSE is expected to be lower. The walking activity achieved a better RMSE than the brisk walking activity. This is also in line with expectations as quicker movements may cause the sensors to move. For identical reasons, we see that the correlation coefficient is maximum for the flexion/extension activity, medium for the walking activity, and lowest for the brisk walking activity.
An example of the results from the flexion/extension activity can be seen in Fig. 6. Flexion angles obtained using the sensor and camera show an excellent match. We see increased error at lower angles which is expected due to the lower dynamic range of the sensor at these angles.
To examine repeatability, we observed the mean absolute difference in RMSE between the first and second trials per activity for each subject. For the walking activity, we observed a mean absolute difference in RMSE of 1.670°. For the brisk walking activity, we observed a mean absolute difference of 1.604°. Finally, for the flexion and extension activity, we observed a mean absolute difference of 1.705°. This indicates the sensors are resilient to being repeatedly taken off and put back on and, therefore, are reliable. In addition, we notice improved accuracy of the sensors over the course of the experiment. The sensors were built with iteratively better quality due to practicing the fabrication procedure. It is expected that if the experiment were reconducted, we would observe even better results.
Discussion
A. Comparison With State-of-the-Art Kinematics Sensors
As seen in Table III, compared to the previous work done on monitoring knee flexion, we achieve similar to slightly higher RMSE values. Specifically, a potentiometer-based approach achieved a 3.35° ± 1.24° RMSE for a walking activity and a 3.64° ± 1.27° RMSE for a running activity [22]. A rotary-position-sensor-based method achieved a 3.0° ± 1.4° accuracy for a walking activity [23]. Another study using an IMU achieved a walking RMSE of 4.3° ± 0.7° and a running RMSE of 7.1° ± 5.4° [24]. Strain sensors show even better results of measuring knee flexion of 1.2°± 0.87° RMSE [25]. Optical fiber sensors have shown to achieve a 5.3° ± 1.13° RMSE during walking [26], while capacitive bending sensors have shown to achieve a RMSE 5.8° ± 1.2° during robotic arm bending tests [27]. Although other approaches may achieve slightly better results, they are bulky, restrict movement, suffer from drift, or are prone to wear and tear. Our proposed sensor solves most of these issues while still maintaining a reasonable accuracy. Notably, in the following section, we reconcile the translation to an untethered, fully wearable version, with a 0.670° ± 0.366° RMSE during phantom bending test.
B. Translation to an Untethered Sensor
Though this proof-of-concept study utilizes a network analyzer to evaluate the kinematics sensor, the end goal is for the sensor to be fully wireless. In this untethered approach, the system would require a transmitting (Tx) board to generate a 34 MHz signal at the desired power level, the loop sensors of Fig. 1, and a receiving (Rx) board to detect the received power level.
As an example implementation, a prototype untethered system was built and tested upon a phantom model. Referring to Fig. 7(a), the Tx board features a CMOS oscillator (SiT8008BC, SiTime) and a fifth order Chebyshev low-pass filter with a 40 MHz cutoff frequency. It was designed to generate an RF power of Pt = 5.68 dBm at 34 MHz upon a 50 Ωload. The Rx board features a bandpass filter operating from 30 MHz to 40 MHz with 10 MHz bandwidth (SXBP-35N+, Mini-Circuits), a logarithmic amplifier (ADL5513, Analog Devices), and a Bluetooth Low Energy (BLE) system-on-chip with an internal 12-bit analog to digital converter (CYBLE-012011-00, Infineon). It was designed to detect the received RF power (Pr), convert the power level to voltage level (Vr), and transmit the voltage data to a remote computer via Bluetooth. With the relationship of flexion angle vs. Vr known through calibration, we can recover real-time flexion angles from the collected voltage data. In this first, proof-of-concept implementation, the footprint of the Tx and Rx boards is 45 mm × 32 mm and, 50 mm × 40 mm respectively.
(a) Block diagram of the proposed untethered sensor. LPF: low-pass filter, BPF: band-pass filter, LA: logarithmic amplifier, ADC: analog-to-digital converter, BLE: bluetooth low energy. PC: personal computer. (b)–(c) test system for untethered flexion angle detection.
The testing setup for the untethered sensor system is shown in Fig. 7(b) and (c). Per studies [28], the leg was emulated as an 8-cm diameter Styrofoam cylinder with a 3D-printed joint used to mimic knee flexion. We note that since biological tissues are non-magnetic, there is no need to include tissue-mimicking materials in the experiment. The loops of Fig. 1 were conformally placed on the phantom and connected to the Tx and Rx boards, respectively. To collect calibration data as well as “gold standard” angles to compare against, we utilized an Intel RealSense 515 LiDAR Camera and placed three markers on the two halves of the phantom and the joint, respectively.
For calibration, we performed slow flexion from 0° to 90° followed by several fast flexion and extension movements over a duration of 60 sec. The purpose of the fast flexion and extension was to align the camera and sensor timestamps and ensure accuracy. Calibration was performed three times for repeatability purposes. Results are shown in Fig. 8(a), confirming repeatability as well as the expected monotonic relationship between voltage and flexion angle.
Experimental results for the untethered sensor system: (a) Calibration curves, and (b) an example recording of received voltage (solid red), angle estimated by the sensor (solid blue), and angle provided by the camera (dashed yellow).
Following calibration, we performed manual flexion and extension of the phantom limb at random speed, over a duration of 80 sec. We used the initial and final 10 sec to align the camera and sensor timestamps and the remaining 60 sec for data analysis. Testing was performed five times for the same reasons as outlined in Section II-E. An example trial is shown in Fig. 8(b). Here, voltage data are plotted as a function of time (solid red), alongside a comparison between the estimated angle from the untethered sensor system (solid blue) and the “gold-standard” angle provided by the camera (dashed yellow). As seen, the estimated angle is in great agreement with the “gold-standard” angle. To quantify performance, Table IV summarizes the RMSE and Pearson's correlation coefficient (ρ) between the estimated angle and “gold-standard” angle for each trail. As seen, the sensor achieves outstanding accuracy with an average RMSE of 0.670° and an average ρ of 0.99966.
Having validated the feasibility and accuracy of a fully wearable sensor system, in the future, we will further miniaturize the Tx and Rx boards, test on human subjects, and optimize the circuit and sensor design accordingly.
C. Clinical Implications
As mentioned in Section I, wearable kinematics sensors have great clinical potential for applications as diverse as prevention [29], rehabilitation [30], [31], and training [31]. Here, we highlight two example applications that we envision our sensors to be utilized in, namely in the fields of:
Anterior Cruciate Ligament Reconstruction (ACLR): Though ACLR surgery improves daily function for individuals with ACL injury, 43% of patients exhibit impaired knee motion [32], [33] and 33% suffer a second ACL injury after return-to-play [34], [35]. Knee motion sets the stage for safe progression after ACLR [36]. However, there is currently no technology capable of remote, high-fidelity collection of knee kinematics during real-world activities. Our sensor aims to fill this gap.
Mild Traumatic Brain Injury (mTBI): Athletes recovering from an initial mTBI are known to be at high risk of subsequent mTBI [37], [38]. Kinematics impairments are viewed as one of the major reasons for such subsequent injuries [37], [39]. However, this relationship is still ambiguous due to the inability of current technologies to measure joint kinematics during on-field athletic activities [37], [38], [40], [41]. Our sensor aims to fill this gap.
Conclusion
This work explored the dynamic motion capture capability of a recently reported class of wearable loop sensors that overcome limitations in the state-of-the-art. We demonstrated real-time dynamic motion capture capability of the sensor using a wearable prototype, provided a calibration mechanism, and provided software for the same. The sensor was validated on ten (10) human subjects for three (3) different types of activities. An RMSE as low as 3.657° was observed, averaged across non-outlier subjects. This is similar to slightly higher as compared to other wearable flexion sensors. Concurrently, the sensors are not confined to contrived environments, are reliable over time and injury-safe, and do not obstruct natural movement. In a major step forward, we further demonstrated proof-of-concept results for wireless boards used to replace the tethered network analyzer setup. An average RMSE of 0.670° was observed upon tissue-emulating phantoms, confirming feasibility for a fully wearable sensor system.
In the future, we will test the fully wearable sensor system on human subjects and improve upon the sensors’ RMSE by further optimizing the design and adopting artificial intelligence techniques to remove fabric drift errors. Ultimately, this sensor can be deployed in numerous settings (clinical, sports, virtual reality, robotics, and more) for real-time motion monitoring, anywhere and anytime.
Author Contributions
Conceptualization, methodology, I.A., C.C., Y.Z., V.M. and A.K.; software, validation, formal analysis, investigation, I.A., C.C., Y.Z. and A.K.; data curation, I.A., C.C. and Y.Z.; writing—original draft preparation, I.A. and C.C.; writing—review and editing, Y.Z. V.M. and A.K.; funding acquisition, A.K.; All authors have read and agreed to the published version of the manuscript.
Conflicts of Interest
The authors declare no conflicts of interest.
ACKNOWLEDGMENT
The authors would like to thank Joe Heiser who supported the design and fabrication of the wireless modules as part of his undergraduate research. The authors would also like to thank Dr. Stephanie Di Stasi and Dr. Jaclyn Caccese from The Ohio State University School of Health and Rehabilitation Sciences for discussions related to the clinical implications of wearable kinematics sensors.