Multi-tap Resistive Sensing and FEM Modeling enables Shape and Force Estimation in Soft Robots

We address the challenge of reliable and accurate proprioception in soft robots, specifically those with tight packaging constraints and relying only on internally embedded sensors. While various sensing approaches with single sensors have been tried, often with a constant curvature assumption, we look into sensing local deformations at multiple locations of the sensor. In our approach, we multi-tap an off-the-shelf resistive sensor by creating multiple electrical connections onto the resistive layer of the sensor and we insert the sensor into a soft body. This modification allows us to measure changes in resistance at multiple segments throughout the length of the sensor, providing improved resolution of local deformations in the soft body. These measurements inform a model based on a finite element method (FEM) that estimates the shape of the soft body and the magnitude of an external force acting at a known arbitrary location. Our model-based approach estimates soft body deformation with approximately 3% average relative error while taking into account internal fluidic actuation. Our estimate of external force disturbance has an 11% relative error within a range of 0 to 5 N. The combined sensing and modeling approach can be integrated, for instance, into soft manipulation platforms to enable features such as identifying the shape and material properties of an object being grasped. Such manipulators can benefit from the inherent softness and compliance while being fully proprioceptive, relying only on embedded sensing and not on external systems such as motion capture. Such proprioception is essential for the deployment of soft robots in real-world scenarios.


I. INTRODUCTION
Soft robots are robotic systems with purposefully designed compliant elements embedded into their mechanical structure.Thereby, soft robots can passively adapt within unstructured environments and are safer when interacting with humans [1].Despite these benefits, instrumenting soft robots to be proprioceptive, i.e., correctly perceiving their own shape [2] in contexts without and with external interactions, remains an open challenge.
The shape of a rigid robot is fully defined by the configuration of its fixed number of joints independent Fr,1 Fr,2 Fr,3 Fig. 1.Advantages of a multi-tapped flex sensor with continuum model.(a) Reconstruction results of the shape of a flexible strip using different sensing and modeling strategies.The dark gray stripes represent true shape of the flexible strips and light gray stripes their estimated shape.The red lines are the flex sensors, the overlaid orange dashed lines indicate multitapping, and the cyan lines represent shape predictions.Dashed cyan outline indicates shape estimated using an FEM-based model.(i) Actual shape of a flexible strip with an embedded flex sensor bent under known forces F r,i .(ii) Given a single flex sensor reading, model with constant-curvature assumption estimates radius of curvature r. (iii) Multi-tapped flex sensor without our modeling approach cannot estimate the shape of the flexible strip beyond the active length of the flex sensor (cyan dotted lines).(iv) Multi-tapped sensing with a model allows the full shape and forces at known points F e,i to be estimated.(b) For a soft pressurized finger, shape estimation results using a single flex sensor with constant-curvature assumption compared to our multitapped sensing strategy with continuum model.
of interactions with one or more loads acting on the system [3].In a soft robot, however, each such interaction creates deformations associated with it.Proprioception in soft robotics, therefore, requires being able to sense these deformations to reason about the state of the system and the load [2] .We consider a suitable performance in proprioception when a soft robot can sense its own configuration within an acceptable error margin, which in this work, we define to be <5% of the characteristic length [3], [4].Furthermore, to be able to estimate load, the approach needs to generalize to the full range of control inputs and states reachable by the soft robot's internal actuation.Thereby, we postulate that highperformance proprioception in soft robotics has to fulfill the following three requirements: (1) high-resolution sensing of local deformations, (2) a model of the soft device, including its inlaid sensor, and (3) a model for the device actuation.
In Sec.II, we shall further discuss these requirements in the context of placing current approaches in soft robot proprioception within this framework and relating them to each other using Fig. 1.The figure highlights the key aspects of our approach and how it relates to other state-of-the-art methods.Following that, Sec.III-A introduces the multi-tapped flex sensor technology and how the shape descriptors (position and angular displacements) are calculated from resistance values.Sec.III-B discusses the flexible strip and soft finger used to validate the proposed proprioceptive approach.Sec.IV describes the finite element model defined in Simulation Open Frameware Architecture (SOFA) 1 .The experimental validation is presented in Sec.V. Finally, Sec.VI summarizes our contributions and provides an outlook to future developments.

II. STATE-OF-THE-ART AND OUR CONTRIBUTION
A. Shape Proprioception with Single Curvature Measurement Several works have already proposed an integrated design and fabrication approach to bring proprioceptive sensors into soft robots [5].Embedded sensors with various sensing modalities can achieve proprioceptive sensing in a soft gripper towards closed-loop manipulation of the object [6], [7].Also, commercially available resistive flex sensors in a soft gripper can detect the object being grasped [8].These approaches, however, fall short in meeting the high-resolution sensing requirement we laid out in Sec.I; they use a single measurement with a constant-curvature assumption as shown in Fig. 1 (ii), i.e., the sensing resolution is low and local deformations are not captured.Also capacitive flex sensors 2 can output coarse shape estimations when local curvatures along the sensor have different signs and therefore cancel each other out [9].

B. Data-driven Force Estimation with Proprioception
Force estimation using proprioceptive sensing can be achieved by either adding force sensors at the tip of the robot [10], [11] or building a robot model based on the Piecewise Constant Curvature (PCC) [12], [13] or Cosseratrod theory [14], [15].Data-driven approaches have also been proposed to address the challenge of force proprioception [16], [17].Recurrent neural networks (RNN) can be used to learn the sensor characteristics of three resistive strain sensors in a pneumatically actuated soft robotic finger and to thereby estimate the external force at the tip of a soft finger [17].This learning approach [17] allows for proprioception with relatively high spatial resolution and helps with issues like hysteresis; however, such approaches fall short of meeting requirements (2) and (3), with an outcome similar to what is illustrated in Fig. 1 (iii).Since it is purely data-driven rather than based on the known mechanical properties of the device, the lack of a mechanical model causes the proprioception not to extend to areas not directly observable by the sensor.

C. Model-based Proprioception
Modes et al. [18] present a model based on the Cosserat rod theory to determine the shape of a sensor array based on accurate longitudinal strain measurements and incorporating bending, twisting, and elongation.Tapia et al. [19] present an approach for optimal sensorization of soft robots using stretch sensors to reconstruct their shape using an FEM-based 1 https://www.sofa-framework.org 2 BendLabs, Inc.
mechanical model and a model for resistive type sensors.In their designs, the sensors cover the whole length of the devices, which allows for finding the deformations due to actuation efforts and external forces.However, the focus is on shape reconstruction, and they stop short of explicitly quantifying force estimation performance or extrapolating shape beyond sensorized regions using their approach.
In our previous work, we introduced a multi-modal method to estimate shape and forces based using FEM [20].Capacitive touch sensing is used to determine the force location on the devices, and air-flow/pressure sensors are used to solve an optimization problem that yields the force magnitudes that best explain the observed changes in sensor values (volume/pressure).This work was followed up by an approach featuring orientation-sensing (one reading per segment) for a fluidically actuated soft arm [21].Here, attention is paid to modeling the effect of actuation and the fiber-reinforcement constraining the deformation of the pressurized chambers.Koehler et al. proposed a model-based sensing method with model-calibrated embedded sensors by solving the inverse problem to find the optimal shape that will lead to the sensor readings [22].However, using high-resolution resistive shape sensing for proprioception (shape and force) has not yet been explored in this framework.

D. Our Contribution
In this work, we use an inexpensive, commercially available 95 mm Spectra Symbol Resistive Flex Sensor.This sensor allows multi-tapping, i.e to make measurements at multiple exposed electrical connections, along the resistive strip.With the additional measurements, the sensor is able to capture local curvature changes for high-resolution shape measurements.The multi-tapped sensor, along with the solid mechanics of the device and the pressure-based actuation are modeled in an FEM-based framework.The modeling allows us to generalize the shape and force estimation to parts of the soft body not directly observable by the sensor.
We address the requirements from Sec.I by contributing (1) the creation of a multi-tapped resistive shape sensor using an off-the-shelf sensor and embedding it in two types of soft devices, (2) the modeling of soft devices with embedded sensors within a finite element modeling framework (SOFA), and (3) modeling within this framework the actuation by pressurization of internal chambers.We embed the shape sensors for validation in two devices, a flexible strip without actuation (Fig. 1 (iv)) and a soft finger with actuation (Fig. 1 (b)).We experimentally validate the proposed approach and show that accurate shape (avg.error < 3%) and force estimation (avg.error < 12%) is possible with the devices (shape for the flex strip and shape and force for the soft finger).However, certain uncaptured nonlinearities still occur, which introduced errors in our system.The model also needs further optimization to run fast enough for real-time estimation.

III. SYSTEM DESCRIPTION
A. Multi-tap Flex Sensor 1) Design and Fabrication: The Spectra Symbol flex-sensor consists of a thin strip of flexible conductive material coated onto a flexible but non-stretchable substrate.When the sensor is bent, the conductive ink is stretched, and its electrical resistivity changes.The resulting change in resistance can be measured to determine the radius of curvature.We multitap this sensor to extend its capability to measure local deformations along its length using a simple flexible printed circuit board (PCB) designed to route individual tapping points to the measurement circuit.A diagram of the PCB layout is shown in Fig. 2. The flexible interfacing PCB is glued to the flex sensor on the side with the exposed pads to make electrical contact using an epoxy glue 3 .There are four simple steps to fabricating the multi-tap flex sensor as shown in Fig. 2. Firstly, prepare the necessary materials: the flex sensor, flexible PCB, epoxy glue, and a 3-D printed mask (0.2 mm thick) with windows that match the exposed pads on the sensor.Secondly, place the mask on the flex sensor, align the windows to the tapping points, and apply the epoxy glue on the mask.Thirdly, remove the mask and check that the epoxy glue is distributed evenly on all tapping points without any bridging between adjacent contact pads.Lastly, place the flexible PCB on the sensor and wait for the epoxy glue to cure.The fabrication process of the multi-tapped flex-sensor is shown in Fig. 2.This design satisfies requirement (1) on high-resolution shape sensing, established in Sec.I. 2) Mapping Resistance to Shape: While multi-tapping allows us to measure changes in resistance across the sensor with better spatial resolution, the mapping from these resistance measurements to the shape of the sensor is not trivial to model.The segments across which we measure resistance are part of the same substrate, and therefore their resistance values are correlated.Local deformation at one end of the sensor still affects the resistance across a segment on the other end, albeit to a lesser extent.Hence, we used a neural network The shape vectors, in our case, are the displacement (translational and rotational) of a set of points matching the placement of the sensor within the soft body.Corresponding points at the same locations are established on the FEM mesh in simulation.A component in SOFA named the pose effector (see Sec. IV-C) serves to apply measured displacements as constraints to an optimization problem, allowing to determine the efforts (internal actuation, external forces) that lead to the sensed shape of the flexible strip or the soft finger (see Fig. 3).
Thus, the shape model of the resistive flex sensor can be expressed as where S is an array of poses that denotes the shape of the soft robotic device where the sensor is embedded at any time.R denotes the vector of resistance measurements corresponding to the sensor segments, and F is the mapping between the two we learn from observations Note that depending on the scenario, S can contain only the translational or rotational portion of the poses.Furthermore, not all degrees of freedom need to be active; for instance, one of the dimensions could be ignored because deformation is only described in a plane.We provide the corresponding details on experimental validation in Sec.V.
3) Nominal Shape Vector: For neural network training, ground truth is established from visual markers whose pose is captured with a camera.The visual markers are attached according to features on the devices that are foreseen during their fabrication (see Fig. 4).Therefore, there is a nominal position for each marker that we can also establish in simulation.
Nonetheless, discrepancies due to fabrication tolerances are still present, which prevents the direct use of the visual marker to feed the simulation.It becomes necessary to establish an additional (barycentric) mapping that converts the measured marker position to its nominal counterpart.
The nominal shape vector S nom is gathered from the FEM model with the following procedure.One point is assumed to be exactly coincident in the camera frame and the simulation, which is a fixed point on the base of the device.This point was chosen as it would be a location least sensitive to fabrication tolerances.Marker positions are captured relative to this base point and initialized directly in the geometric model.We denote the initial marker positions as p 0 .Then we select a set of evenly spaced predetermined locations on the geometric model, which we denote as S 0 .These points correspond to the position of the sensor within the device for the shape vector S nom .The marker positions and shape vector are connected to the mesh via barycentric mapping.Therefore, as the device deforms, the relative poses between the markers and the nominal poses are kept coherent through the mechanical model.The nominal shape vector S nom is the output of the sensor regression and is fed to the simulation.

B. Flexible Strip and Soft Finger
The proposed concept is evaluated using the hardware and systems shown in Fig. 4. Overall, the system is based on a multi-tapped resistive flex sensor.The sensor was integrated into different devices to investigate its sensing capability.Additionally, measurements of visual markers placed on the soft body serve as ground truths in the sensor model training.
With the trained sensor model, the requirement (1) from Sec.I can be fulfilled.To test the multi-tapped flex sensor's capabilities, we embedded it into a 3D-printed flexible thermoplastic polyurethane strip (Ultimaker TPU 95A) with a narrow slit in the middle designed to insert the multi-tapped flex sensor.Furthermore, the multi-tapped sensor is embedded in a soft finger 3Dprinted in an elastomer material which is part of a soft gripper designed by Festo SE & Co. KG.The finger comprises a chamber for pneumatic actuation and an embedded metallic strip to act as a constraint layer for bending.However, the finger is relatively stiff, and the shape-sensing capabilities of the multi-tapped sensor cannot be fully investigated with this device.Therefore, we introduced the flexible strip to test shape-sensing.
We used the SOFA framework and its Soft Robots plugin [23] to build a model based on the finite element method.The simulated model running in SOFA is updated according to the sensor values, yielding a shape estimate (see Fig. 4 (a) and (b)).After calibration using a force-torque sensor, SOFA is able to represent both the effect of known pressurization as well as unknown external forces (see Fig. 4 (b)) without the need for any additional sensing modalities.Thereby the simulation model satisfies requirements (2) and (3) from Sec. I.

A. Equations of Quasi-Static Motion
We use an FEM mesh which yields the internal elastic forces F(q), given that the nodes are at positions q.We choose Hooke's law to describe the relation of force and deformation, i. e. the material properties are characterized by Young's modulus and Poisson's ratio.At each step i of the simulation, we compute a linearization of the internal forces as follows: where dq i = q i − q i−1 is the node displacement and K = ∂F(q i−1 ) ∂q is the tangential stiffness matrix for the current node positions q.Now, we add external forces to the equation for a complete picture: H T λ is a vector that gathers boundary forces, such as contacts or external controlled inputs.The size of λ is equal to the number of rows in H and to the number of actuators (contact forces, cables, etc.).P represents known external forces, such as gravity.Then, (3) is solved under the assumption of a static equilibrium at each timestep, i. e. the left-hand side is 0, as the effects of acceleration can be neglected in our setting.This delivers a motion that is a succession of quasi-static states.

B. Force, Pressure, and Length Constraints
In our FEM-based simulation, the constraints include an external force constraint, a pressure actuation, and a length constraint.As discussed in Sec.IV-A, constraints are gathered in H T and λ in Equation (3) [23].To find these two terms, we need to define the simulation boundary conditions.
The flexible strip is the simpler case, only requiring external force constraints.The external force constraint directly specifies the constraints in the task space.They are defined by a force magnitude λ and a direction at a certain point p i on the mesh encoded by H i .The soft finger is additionally actuated by the pressurization of its internal chamber.Thus, the deformation of the soft finger is affected by the deformation of the chamber.The Lagrange multiplier λ represents the pressure on the surface of the chamber, while H is related to the normal direction of the vertices of the mesh defining the surface of the chamber.Another major component affecting the dynamics is the combination of the metallic strip and the multi-tapped flex sensor embedded in the finger.Since the metallic strip and the flex sensor are fixed at both ends, they constrain the finger's elongation on the front side, the one opposite to the chamber.We model this constraint using a length constraint with a sequence of fixed-length segments that follow along the length of the metallic strip.The Lagrange multiplier λ, in this case, represents the force necessary to keep the length of each of the segments in the sequence constant.H is built using the constrained direction as if the sequence represented an inextensible cable [24].Fig. 5 illustrates the external force constraints.Their relationship to solving an inverse problem for a given sensor configuration is illustrated in Fig. 5 and explained in the next subsection.

C. Finding Constraining Forces Through Optimization
We use inverse problem solving [23] to optimize for the generalized actuation forces λ that will minimize the differences between the simulated shape vectors and the sensed shape vectors.This means we search for actuation efforts that will best explain the observed deformation.We consider two types of forces: unknown external disturbances from pushing and known actuation by pressurized air in the chamber of the soft finger.In SOFA, the optimization is solved for each time step.Thus, in practice, the problem is solved by finding ∆λ that corresponds to the difference in actuation forces that will minimize the error with regard to the change in shape for that time step.External forces on the real device deform the flex sensor (red), which is captured as a change in shape ∆δ real = (g 1 − e 1 , . . ., gn − en) T .SOFA finds the forces ⃗ F ∆λf i applied at points p i that minimize the distance of the effector points to the goal points.The forces bring about a change of shape of the whole device, including the points p i that move to their new positions p ′ i .Note that the forces do not necessarily need to be applied on the section that is covered by the flex sensor, as illustrated here.
δ e := δ e (q) is a function that maps the positions q of the FEM nodes to a pose of interest to the user, which is called a pose effector.The pose in this case, could be cartesian coordinates only (3D) or include orientation as well (6D).A displacement ∆δ e,i of an effector i is composed of a positional and angular displacement (represented as angular velocity): ∆δ e,i = (∆x, ∆y, ∆z, ω x , ω y , ω z ) T .To find the relation between the change in applied forces and the change in shape in the simulation ∆δ e,sim of a collection of position effectors, we can write: where W ef := H e K −1 H T f is the matrix that directly maps changes in actuation force to changes in the effector displacement (position and angular).This matrix can be obtained based on the constraints defined in Sec.IV-B.
Then the optimization problem can be formulated as follows: where the ∆δ real = g − e = (g 1 − e 1 , . . ., g n − e n ) T = ∆S nom are the real changes in the shape (see also Sec.III-A2).Furthermore, actuation efforts can also be taken into account during optimization by further constraints, such as The equality constraints model that some of the efforts ∆λ are known actuation of the pressurized chamber, as it is the case for the finger.The quadratic program (QP) is solved using a Linear Complementarity Constraint Solver [23] and we obtain a value for all ∆λ and, in particular, for the external forces ∆λ f due to the pushing.
Therefore, the requirements ( 2) and (3) established in Sec.I are met with a model capable of dealing with the solid mechanics, the sensing, and the actuation of the devices.

V. EXPERIMENTAL SETUP AND RESULTS
A. Data Collection 1) Flexible strip: We first tested our proposed method using the flexible TPU strip described in Sec.III-B.We attach a series of 11 markers along the narrow side of the flexible strip for ground truth measurement.The flexible strip contains the sensor with an active length of 80 mm.Note that the flexible strip is deliberately made to be longer (120 mm) than the sensor's active length, such that the multi-tapped sensor can only measure the shape of the strip corresponding to the first 8 markers (out of 11 total).The shape of the rest of the flexible strip will be inferred from the shape of the directly observable section of the strip using the FEM model without needing to retrain the neural network with this data.The flexible strip is then attached rigidly at the base and deformed arbitrarily by hand to collect data.The training, validation, and test sets are recorded separately, where we deform the strip and log the resistance change and shape of the strip.For this device, we are interested in evaluating the accuracy of the shape reconstruction with the help of the solid mechanics model in SOFA.
2) Soft finger: Now we test our method on a pneumatically actuated soft finger.The active length of the finger is 85 mm.Similar to the flexible strip, a series of markers are attached to the side of the finger for ground truth measurements.The finger is actuated with different pressures ranging from 0 kPa to 1.8 kPa, and an external force is applied on the finger by hand to collect training data.Note that in this case, the forces applied are not measured, and only the shape of the finger and corresponding resistance values are recorded.
The validation and test sets are recorded separately, where we use a robotic arm 4 to apply an external force at a known location on the finger.The robotic arm has a 6-axis forcetorque sensor to measure the force applied at the end-effector.The applied force is perpendicular to the contact surface, limiting the force to a single axis.Therefore, the measured force is uniaxial along the normal axis of the contact surface.
We also use the validation set to identify the parameters for FEM (see Sec. V-C).Actuation pressure for validation set and test set are 1.5 kPa and 1.2 kPa, respectively.The external forces have range between 0 N to 5 N and are applied at different locations along the finger.For each dataset, the data collection starts when the soft finger is fully pressurized, the robot arm approaches the soft finger and touches the finger's inner surface.Then, the robot arm gradually increases and decreases the force magnitude while applying the force perpendicular to the contact surface.After the contact force ramps down back to 0, the robot arm moves away from the finger and data collection stops.We repeat this process with different pressures and different force magnitudes to construct the dataset.For this device, we are interested in evaluating the accuracy of not only shape reconstruction but also the magnitude of the external force estimation.

B. Resistances to Shape Regression
The mapping from resistances to shape is modeled as described in Sec.III-A.We will now focus on specific details on the neural network training.Firstly, the data we collected was filtered and downsampled to 30 Hz to synchronize with the vision measurements.In this dataset, unactuated neutral states dominate the data collection process, and deformed states are under-represented.The training data is, therefore, resampled to improve the skewness of the dataset.Nominal shape vectors are then computed from the measured marker positions using the FEM model.We used orientation change as the shape vector for the flexible strip and position changes for the soft finger.This difference comes from the different constraint settings in the two cases.The soft finger is constrained to have a constant length to ensure it bends under pressure actuation, while the flexible strip is not.Therefore, flexible strips are more sensitive to position errors.To avoid numerical instability, we used orientation change as the shape vector.Then we train the sensor model from this data.
For shape estimation of the sensor embedded in the flexible strip, we used a convolutional neural network with the input being a matrix containing the resistance vectors R t stacked as follows: where t refers to 4 Flexiv Rizon 4s the timestep.The model consists of a convolutional layer of 32 3×3 filters with 0.5 dropout, flattened and fed to an output layer that predicts the relative position of each shape vector with respect to the previous one.The data collection procedure was run at 30 Hz for 10 min to explore the state space and train this neural network.However, in the case of the soft finger, the motion is more limited, and the state space that is being explored is limited as well.Therefore we chose a less expressive network with two fully connected layers of 32 and 16 neurons with 0.2 dropout.Input to the model is a vector of resistance measurements made at the current timestep.The data collection procedure was run at 30 Hz for 15 min to explore the state space and train this neural network.

C. Parametric Identification for the FEM model
For the flexible strip, the FEM model predicts the shape evolution beyond what the flex sensor can estimate.Since it is a purely geometrical problem, no material parameter identification was performed.We chose the model parameters based on knowledge of material properties such that the deformation is feasible and the system converges.For the soft finger, the FEM model serves as a dynamic model for force estimation, and the parameters need to be calibrated experimentally, including the Poisson ratio and Young's Modulus.In both cases, mesh density was chosen based on trial and error to match reality while maintaining a reasonable framerate for the simulation.The material is modeled as quasiincompressible, and the Poisson ratio is set to 0.45 [25].An approximate value of Young's Modulus is obtained by minimizing the position error with different actuation inputs ranging from 0 kPa to 2 kPa with no external disturbances, which is determined to be 1.37 MPa.Additionally, for force estimation, an empirical scaling factor of 1.5 is introduced to scale Young's Modulus and actuation pressure simultaneously to account for errors in the modeling process.This scaling factor is obtained by minimizing errors in force estimation in the validation set.The scattered points are the marker locations and the color scale represents the error magnitude.Markers are labeled according to the distance to the reference point to explain the observed deformation.For the shape estimation of the flexible strip under deformation, a series of six external forces are arbitrarily applied at uniformly spaced locations along the strip.The results of the shape estimation of the flexible strip are summarized in Fig. 6 as reconstruction error statistics.As mentioned previously in Sec.V-A1, only the first 8 points are covered by the sensor, and the rest are calculated by the FEM model based on the mechanics of the device.The error is normalized by its distance to a fixed reference point at the base.The error increases over the length of the device because the error accumulates over each marker.At the last point covered by the sensor, the average error is 2.95% with respect to its distance to the reference point.At the end of the flexible strip, where the sensor is not present, the average error is 3.61% with respect to its distance to the reference point.The ability to make an estimation at the location, where the sensor is not present, demonstrates one of the advantages of the FEM model in comparison to a data-based model for extrapolating a system (requirement (2)).
2) Shape and Force Estimation with the Soft Finger: Fig. 6 summarizes the shape reconstruction error for the soft finger in the test set.Similar to the flexible strip, the error increases over the length of the finger.At the furthest point from the base, the average error is 2.82% with respect to its distance from the reference point.Error distributions at different locations are illustrated in Fig. 7.
The reconstructed shape of the finger is then used to estimate the external force.Results from a force estimation experiment are shown in Fig. 8.We used shape measurement from both camera and resistive sensors to investigate how much error is introduced from the sensor mapping.The estimated force using the camera follows the measured force better without the error of the additional mapping.The average relative errors are 10.18 % and 11.82 % with respect to the force range for vision and resistive estimations, respectively.
3) Discussion of the Experimental Results: Our position estimation reaches around 3% error with respect to the total length within the sensor coverage, which is a substantial improvement over the results reported in [17] with a flex measured forces are obtained using a force-torque sensor behind a pointed tip, see Fig. 4 (c).The estimated forces use a FEM model and either a shape estimation from a multi-tap flex sensor or a visual shape measurement.
sensor.The errors are increasing as the measurement point moves away from the reference point.The errors are not consistent during the experiments, as shown in Fig. 7.The error reaches its maximum in the middle and furthest locations, which we believe is caused by the nonlinearity that our model fails to capture.Apart from the nonlinearity, the dynamic motion of our experiment will also contribute to the model's error, which can explain the error in the middle.Furthermore, the performance of our sensors is comparable to the cPDMS sensors Thuruthel et al. proposed without the need to fabricate a customized integrated sensor for individual robots.Additionally, we were able to achieve a much larger range and a slightly better relative error for the magnitude of the force estimation.The range of the force being estimated in our model-based force estimation is 5 N with approximately 11% relative error, and the range reported using a data-driven approach is 0.5 N with 15.3 % relative error [17].In our experiments, the estimations are higher than the measurements after reaching the peak.We believe such behavior is related to hysteresis in the resistive sensor that is not adequately modeled by the neural network.The advantage of the modelbased approach is the generalization ability to different object shapes and locations of action without training a new model.An advantage of the data-driven approach is the speed of inference.For example, the model in [17] was tested in realtime at around 10 Hz.Our model for flexible strips was run in real-time at 30 Hz, but the structural complexity of the soft finger requires a finer mesh for accurate simulation, and therefore, the force estimation currenly runs at around 5 Hz.

VI. CONCLUSIONS AND FUTURE WORK
In this work, we identified and addressed three key requirements for enabling proprioceptive shape and force sensing in soft robotics.The first is a technology that allows high-resolution shape sensing for capturing local deformations.The second is the use of a model that estimates forces based on the solid mechanics of the soft robotic device.The third is the ability to integrate and account for known actuation inputs in the model during force estimation.Based on these requirements, we proposed a multi-tapped resistive and modeling-based approach to expand the sensing capabilities of a widely available off-the-shelf resistive flex sensor.We employed a neural network to interpret the nonlinear, coupled behavior of the multi-tap sensor readings.The resulting sensor signals informed an FEM-based model of the soft finger that also includes fluidic actuation.Besides the shape, we also estimated an external force acting on the body.The range of the external force was tested for 0 N to 5 N at arbitrary known locations.Our results show the potential of a multi-tapped resistive flex sensor when used with FEM-based modeling in a physical soft robotic gripper for shape reconstruction and force estimation.We believe this approach will bring us closer to proprioceptively grasping with soft grippers in unstructured, human-centric environments.
We propose for the future to further explore the potential of resistive proprioception.First, the sensor itself can be customized and fabricated with a built-in multi-tap interface, which allows for widespread adoption.Second, we would like to optimize our FEM model for force estimation to achieve real-time performance, for instance, using a datadriven approach on the FEM model towards model order reduction [26].Third, we would like to implement a numerical model of the multi-tapped sensor in SOFA so that we can directly map the measurement into the simulation and solve for the optimal solution.Lastly, we would like to investigate the possibility of contact detection and thereby force localization through shape estimation.

Fig. 2 .
Fig. 2. Fabrication and design of the multi-tap flex sensor.(a) Diagram of the flexible interface PCB (b) Fabrication process of the multi-tap flex sensor.(i) The flex sensor and a 3D-printed mask.(ii) Apply conductive epoxy to the flex sensor using the mask.(iii) Conductive epoxy applied on the exposed contact points is visible after the removal of the mask.(iv) Flexible interfacing PCB is attached to the sensor and held together by the conductive epoxy.(c) Structure of the multi-tap flex sensor.

3Fig. 3 .
Fig. 3. Structure and computational model of the soft finger.(a) Structure of soft finger.(b) Design of the soft robot finger.(c) Diagram of the FEM model for the soft finger.Boxes represent correspondence in design and the FEM model.

Fig. 4 .
Fig. 4. Multi-tap sensor and soft devices tested with their FEM models.(a) Flexible strip deformed under external force (left) and its simulation in the SOFA (right).(b) Pressure-actuated soft finger deformed under external force using a force-controlled robotic arm (left) and the finger's simulation in SOFA (right).

Fig. 5 .
Fig. 5.The constraints applied to the FEM model of the flexible strip.External forces on the real device deform the flex sensor (red), which is captured as a change in shape ∆δ real = (g 1 − e 1 , . . ., gn − en) T .SOFA finds the forces ⃗ F ∆λf i applied at points p i that minimize the distance of the effector points to the goal points.The forces bring about a change of shape of the whole device, including the points p i that move to their new positions p ′ i .Note that the forces do not necessarily need to be applied on the section that is covered by the flex sensor, as illustrated here.

Fig. 6 .
Fig.6.Reconstruction error for the flexible strip and soft finger.On the x-axis is the distance along the length of the body from the fixed base.On the y-axis is the position error w.r.t vision-based ground truth measurement normalized by its distance to a fixed reference point at the base of the object.

D. Experimental Results 1 )Fig. 7 .
Fig. 7. Error distributions in the estimation workspace of the soft finger.The scattered points are the marker locations and the color scale represents the error magnitude.Markers are labeled according to the distance to the reference point

Fig. 8 .
Fig. 8. Measured and estimated forces for the soft pressurized finger.measuredforces are obtained using a force-torque sensor behind a pointed tip, see Fig.4 (c).The estimated forces use a FEM model and either a shape estimation from a multi-tap flex sensor or a visual shape measurement.