A Tactile Sensing Concept for 3-D Displacement and 3-D Force Measurement Using Light Angle and Intensity Sensing

This article proposes an optical-based tactile sensor design concept, which uses a light angle and intensity sensor to infer force and displacement from deformations of a silicone pillar. The proposed design uses a simple, low-cost fabrication method with an overall small-scale form factor. The sensor can measure 3-D displacement, 3-D force, and vibration. The overall displacement estimation error [mean ± standard deviation (SD)] in the <inline-formula> <tex-math notation="LaTeX">${X}$ </tex-math></inline-formula>-, <inline-formula> <tex-math notation="LaTeX">${Y}$ </tex-math></inline-formula>-, and <inline-formula> <tex-math notation="LaTeX">${Z}$ </tex-math></inline-formula>-axes was <inline-formula> <tex-math notation="LaTeX">$-10.8\,\,\pm $ </tex-math></inline-formula> 52.1, 15.4 ± 66.5, and 1.8 <inline-formula> <tex-math notation="LaTeX">$\pm 17.6 \mu \text{m}$ </tex-math></inline-formula>, respectively, over a full-scale lateral displacement of 1-mm radius in <inline-formula> <tex-math notation="LaTeX">${X}$ </tex-math></inline-formula> and <inline-formula> <tex-math notation="LaTeX">${Y}$ </tex-math></inline-formula> and 2.2-mm compression in <inline-formula> <tex-math notation="LaTeX">${Z}$ </tex-math></inline-formula>. The overall force estimation error (mean ± SD) was <inline-formula> <tex-math notation="LaTeX">$-8.5\,\,\pm $ </tex-math></inline-formula> 47.6, <inline-formula> <tex-math notation="LaTeX">$-8.5\,\,\pm $ </tex-math></inline-formula> 49.0, and <inline-formula> <tex-math notation="LaTeX">$-28.0\,\,\pm $ </tex-math></inline-formula> 92.6 mN for a full-scale force of approximately 2 N in <inline-formula> <tex-math notation="LaTeX">${X}$ </tex-math></inline-formula> or <inline-formula> <tex-math notation="LaTeX">${Y}$ </tex-math></inline-formula>, and 6 N in <inline-formula> <tex-math notation="LaTeX">${Z}$ </tex-math></inline-formula>. Sensitivity to vibrations in the range of 10–950 Hz was also evaluated showing good sensitivity over this entire range. This new sensing approach could be of benefit in robotic manipulation applications, as it could be easily arrayed and/or integrated into the fingers of a robotic gripper to sense slip events and measure load and grip forces and torques.

R OBOTIC grippers and prosthetic hands have yet to achieve human-like dexterity in object manipulation in part due to a lack of adequate tactile sensation [1].As robots are put to work in unstructured environments, tactile sensation becomes increasingly important for dexterous object manipulation, as such manipulations require accurate information about the contact state of the interface between the robotic gripper and the object.Similarly, humans struggle to perform basic gripping tasks effectively when the sense of touch is removed [2], highlighting that tactile feedback is necessary to maintain grasp security during object manipulation and avoid dropping objects.Therefore, to allow robotic grippers The authors are with the UCD School of Electrical and Electronic Engineering, University College Dublin (UCD), Dublin, D04 V1W8 Ireland (e-mail: stephen.redmond@ucd.ie).
Digital Object Identifier 10.1109/JSEN.2023.3293967 to achieve dexterous manipulation of objects, it is necessary to equip grippers with tactile sensors, the design of which may be inspired by the human sense of touch.This would provide essential tactile information to enhance a robotic gripper's ability to grasp objects successfully and work in unstructured environments [3].In addition, for the tactile sensor to be useful, it must record relevant tactile information while being embodied in a size compact enough to be integrated into a robotic finger.
In the skin of the human finger pad, there is a dense array of mechanoreceptors that encode the information about the contact, material properties, forces, torques, slippage, and friction of the contact interface [4].An enormous variety of tactile sensors have been designed to mimic the human finger pad, transducing pressure, normal and shear forces, and, more recently, detecting incipient and gross slip at the sensor-object interface.
Capacitive sensors have a high spatial resolution and large dynamic range but can suffer from noise interference due to field interaction, meaning complex circuitry is required for filtering [7], [10], [11].Huh et al. presented a capacitive array forming a tactile sensor in which deformations of an array of individual nibs alter the capacitance between four electrodes and an embedded conductive fabric beneath the array.From this, normal and shear pressure, as well as directional vibrations, are inferred [10].However, the sensor uses two different methods that only work separately to perform these measurements.One method is limited to measuring localized normal and shear forces, while the other is a dynamic reconfiguration of the sensing cluster to mimic the human mechanoreceptors [10].
A compact, three-axis magnetic-based sensor, uSkin, measures 3-D forces and has been integrated into multiple robotic systems [12], [13], [14] because of its compact design and modular sensing modality.However, magnetic sensors can suffer from external magnetic interference, either from external magnetic fields or distortion of the field created by the sensor's permanent magnet(s) due to soft iron effects from materials in the local environment.
Optical tactile sensors have been very popular in recent times.Optical tactile sensors utilize the properties of light to transduce force and/or displacement.These methods have been attracting significant attention due to their advantages of achieving high spatial resolution and generally excellent resilience to electromagnetic interference.Optical tactile sensors can be broadly divided into two groups based on whether they use a camera.

A. Camera-Based Optical Tactile Sensors
Many tactile sensors use a camera to transduce tactile information, with most approaches using tracking markers on the sensor skin to estimate skin deformation [5].Most earlier camera-based tactile sensors used 2-D images to reconstruct the 3-D displacement of tracking markers that are patterned on the skin [8], [15], [16].This approach results in a limited ability to sense deformation or force applied normally to the sensor skin.Color gradient mapping can help solve this problem, as this method allows the measurement of surface relief patterns, hence allowing the estimation of the applied normal forces/pressures.However, it leads to a significant increase in size, as it requires multiple light-emitting diodes (LEDs) [17].The GelSight sensor can obtain high-resolution tactile images of the 3-D contact surface topography [8].Nevertheless, it has a large overall size resulting in the motivation to develop the finger-shaped GelForce sensor, which has an incredibly high spatial resolution (0.024 mm/pixel) and is more successful in robotic tasks [18].This smaller version of the GelSight sensor has a height of 25 mm [9], [19], but comes at the cost of reduced accuracy of displacement measurements normal to the skin surface due to the nonuniform illumination of the skin required to achieve the size reduction [9].Dong et al. improved the fingertip GelSight sensor by reducing the reconstruction errors.However, the camera was positioned 35 mm away from the sensor due to the focal length, increasing the overall size of the sensor [8].Using the GelSight principle, the curved OmniTact was developed, allowing multidirection high-resolution sensing with a small form factor of 30-mm diameter and 33-mm height [15].Multiple endoscope cameras were required to achieve this, making the overall sensor incredibly expensive to produce [15].The TacTip also uses a camera to detect the properties of the contact between the sensor and an object using the relative displacement of different regions of the contact area [16], [20].Reducing the size of camera-based sensors may require sacrifices in the accuracy of the displacement/force estimates parallel to the optical axis of the camera lens [5].Indeed, the focal length required to have a clear image of the skin means camera-based sensors are fundamentally bulky.
Camera-based sensors have a great spatial resolution, especially for X Y deformations.However, they have an inherently poor temporal resolution when standard cameras are used due to having to sample 100 000s or millions of pixels, with frame rates in the range of 30-120 frames/s (FPS).This can lead to rapid transient or periodic high-frequency information being lost, such as vibrations caused by slip events.Event-based cameras, which can achieve incredibly high frame rates in relative terms, could solve the temporal resolution problem, but such technologies are still very expensive.

B. Non-Camera-Based Optical Tactile Sensors
Non-camera-based optical tactile sensors are gaining attention due to their potential to be very low profile (small height/thickness), be distributed over large or non-flat surfaces, have low postprocessing requirements (no computer vision required), and have a very high temporal resolution.In addition, eliminating the use of a camera in the design of an optical tactile sensor allows the sensor to be much more compact and more integrable into robotic gripping systems.A typical working principle of a non-camera-based optical tactile sensor utilizes the detection of a change in light intensity by a photodetector [6] when a contact force occurs, which allows tactile information to be extracted from these changes in measured intensity.Tactile optical-based sensors take a variety of shapes and use different materials, such as elastomers, optical fibers [21], photodiodes [6], and liquid lenses [22].
An early non-camera-based optical tactile sensor, commercialized by the company Optoforce (subsequently acquired by OnRobot), uses multiple photodiodes and a reflective layer to create a robust 3-D force sensor and is commercially used in many applications, including assembly lines and handling applications [23].
The use of optical fiber to create tactile sensors is becoming more popular due to their small thickness (<3 mm) and flexibility, meaning they allow the creation of versatile robotic or electronic skins [24], [25], [26].More recently, they have been shown to be able to measure pressure, vibration, and force vectors and be thermosensitive with a high signal-tonoise ratio and repeatability [25], [26].These sensors utilize optical micro/nanofibers to guide light along a path and transduce properties of soft polymers via power loss detected from the deformation and have been successfully integrated onto a robotic hand to enable adaptive grasping [26] and integrated into a wearable glove [25].
A friction-based tactile sensor (PapillArray) developed by Khamis et al. [27] uses LEDs and photodiodes to detect displacement of individual tactile sensing elements and has been used for grip force control [6], [28].However, the design principle, which employs a camera obscura and quadrant photodiode to image the movement of a pillar from the inside, may be challenging to miniaturize using low-cost manufacturing approaches, restricting the sensor's versatility.
In this work, we present a new, optical-based, small-scale proof-of-concept tactile sensor that can estimate 3-D point displacements and 3-D point forces by interpreting the measured angle and intensity of light [i.e., the light vector (LiVec)] arriving at an optical sensor; thus, we named it the LiVec sensor.Compared with other existing optical-based tactile sensors [8], [22], [29], [30], the LiVec sensor has a simple structure consisting of a soft deformable skin, LEDs, and a light angle and intensity sensor.All the sensing elements are combined onto a singular printed circuit board (PCB), giving it the potential for substantial volume reduction compared with other existing tactile sensors [6], [9], [20], [27], [31], [32], [33], [34] and significantly reducing the complexity of the manufacturing process.The design of the LiVec sensor brings the advantages of low-cost off-the-shelf components, a simple mechanical structure, and a small form factor, making it well suited for integration with existing robotic grippers.

A. LiVec Sensing Principle
The LiVec sensor has a deformable skin, placed on a PCB, which encompasses the electronics sensing components.The transduction principle for the LiVec sensor is as follows: light emitted by two infrared LEDs illuminates a white diffuse reflector embedded at the top of a cavity inside a black silicone pillar [Fig.1(a)].The silicone pillar is black to minimize other light reflections, such that most reflected/scattered light originates at the white diffuse reflector [Fig.1(b)]; the black silicone also helps reduce interference from light sources external to the sensor.Since the 3-D position of the internal white reflector is affected by pillar deformation due to applied external forces, the displacement of, or force applied to, the external tip of the pillar can, thus, be inferred from the light reflected off the internal white reflector [Fig.1(c) and (d)].Say photons, primarily being reflected from the diffuse reflector, arrive on the detector of the light angle sensor (ADPD2140, Infrared Light Angle Sensor, Analog Devices) with a unit direction vector u = (u x , u y , u z ) (the LiVec direction shown in Figs. 1 and 3 is an illustration of the LiVec configurations).The light angle sensor will output two photocurrents, which differentially encode the statistics of the X angles of all photons incident on the X -angle sensing regions of the detector.The relationship among the X angle, arctan(u x /u z ), and the two photocurrents is nonlinear and is described in the ADPD2140's specification document [35].The light angle sensor also outputs two additional photocurrents, which differentially encode the aggregate statistics of the Y angles, arctan(u y /u z ), of all photons arriving at the Y -sensing areas of the detector.All four photocurrents are also proportional to the number of photons incident on the detector area.Thus, as the reflector moves closer to the light angle sensor, the amplitude of the four photocurrents typically increases [Fig.1(d)].Therefore, the 3-D displacement of, and force applied to, the external tip of the pillar can be inferred from the The light, which reaches the pillar cavity walls, is mostly absorbed by the black silicone.Light reaching the reflector is diffusely reflected, with some of that scattered light arriving at the detection area of the ADPD2140 light angle sensor.Note that, it is not trivial to illustrate that light reflecting from all parts of the disk-shaped reflector is incident on all parts of the light angle sensor detector area, but the average incident angle of the light when the sensor is in this neutral position is 0 • (relative to the normal to the PCB); again, in this illustration, only some light rays are shown for clarity.(c) Illustration of a lateral displacement of the external pillar tip in the X-direction by deforming the dome-shaped pillar.The average incident angle and intensity of photons arriving at the ADPD2140 light angle sensor change with the reflector position.(d) Illustration of Z compression with no lateral displacement.The reflector is nearer to the ADPD2140 light angle sensor, and so, the average incident light intensity on the light angle sensor is increased.Therefore, intensity changes can be used to help infer Z displacement.
four photocurrents output by the ADPD2140 light angle and intensity sensor chip.To convert the photocurrents (a proxy of light angle and intensity statistics) into displacement and force estimates, a calibration experiment was done.A multivariate polynomial regression is performed on the resulting calibration data to acquire six regression equations that each take as input the (re-)biased four raw photocurrents and their sum (i.e., five input variables in total) to compute estimates of the X Y Z displacement and force.These calculations are later applied to novel photocurrent readings to obtain the real-time estimates of the displacement and force.

B. Simulation of Received Signal Power for One and Two LEDs
Before manufacturing the LiVec sensor, we wanted to understand if a single LED would be sufficient to illuminate the reflector for all pillar deformations, such that an adequate signal-to-noise ratio was achieved, or whether two LEDs might be required.Therefore, the mechanical and optical behavior of the LiVec sensor was investigated in simulation.A simulation model similar to that presented in [36] was developed to explore the optical transduction method.Finiteelement analysis in Ansys Workbench (Ansys, USA) was used to obtain the deformed pillar configurations.The pillar and reflector were assigned neoprene rubber material properties, Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.
and the plate making contact with the tip was configured to be plastic (ABS).A frictional contact was established between the pillar and the plate, using a static friction coefficient of 0.8, while a bonded contact was made between the pillar and the reflector.For the boundary conditions, the plate on top of the pillar was assigned a fixed support, and the base of the pillar structure was subjected to a prescribed displacement.The pillar was first moved in the negative Z -direction against the plate to reach a set compression level (−0.5, −1, or −1.5 mm), and then moved laterally in either the positive or negative X -or Y -directions by 1 mm.Each of the pillar configurations was then imported into LightTools (Synopsys, USA), so the light reflection inside of the cavity could be simulated to obtain an estimate of the incident angle and intensity of the light received at the light angle sensor.The receiver surface was modeled as a 0.31-mm 2 surface placed at the center of the light angle sensor, based on the radiant sensitive area listed in the ADPD2140 specification [35].The LED emitters were designated transmitting optical properties, and their emittance was set to 0.05 W, which was taken from the chosen infrared LED data sheet [37].The reflector was assigned with Lambertian scattering optical properties with ideal (100%) reflectance.The remaining materials in the simulation were set as perfect optical absorbers.Light simulations were run for each X Y Z displacement of the pillar configuration described above.
Fig. 2(a) shows a deformed pillar configuration for the simulation of the LiVec sensor.We can observe how the light gets emitted from the LEDs on both sides of the light angle sensor, goes up to the reflector, and is reflected/scattered back down with the relevant incident rays collected at the simulated light angle sensor.Fig. 2(b) summarizes the received incident light intensity for a 1-mm pillar deformation in the negative versus positive X -direction (the direction for which the distance between a single LED and the reflector would vary most) for three difference Z compressions (−0.5, −1, and −1.5 mm).As expected, there is an asymmetry in response to negative versus positive X movement when using one LED, which is not necessarily an issue.However, when using only one LED [1 LED in Fig. 2(b)], the received light intensity can be as little as 0.2 W/sr, when the reflector is moved away from the LED in the negative X -direction.Thus, for the proofof-concept prototype design proposed in this article, with the purpose of reducing risk in the design, we chose to use two LEDs to ensure a better signal-to-noise ratio.

C. LiVec Design
The LiVec sensor prototype presented in this article [Fig.3(a)] comprises of a PCB, a light angle sensor (ADPD2140 Infrared Light Angle Sensor, Analog Devices), two infrared LEDs (IN-S42CTQIR, Inolux Corporation, Santa Clara, CA, USA), a molded silicone skin with a pillar protrusion that contains a hollow internal cavity, a reflector embedded in the pillar cavity, and top plate used to attach the silicone skin to the PCB [Fig.3(b)].The silicone pillar is cylindrical with a hemispherical tip and was fabricated with an overall height of 9 mm and an outer diameter of 8.5 mm.The pillar contains a cylindrical cavity and side skin thickness of 1.0 mm, making the inner diameter of the cavity 6.5 mm, which encloses the LEDs and the light angle sensor.A circular reflector with a diameter of 1.5 mm made of white silicone is poured into a hole in the black silicone at the top of the internal cavity of the skin.It serves as an indirect light source (scattering the LED light), with its 3-D position influenced by the deformation of the pillar due to applied external forces.The skin color was specifically chosen to be black with a white reflector, so that the majority of the reflected light comes from the reflector to improve sensing precision.This LiVec sensor prototype has an overall height of 11 mm from the back of the sensor to the external tip of the silicone pillar of the LiVec sensor.
The prototype presented here is a proof-of-concept design that aims to demonstrate the LiVec sensing principle, meaning that the size and shape of the pillar protrusion were chosen to prove the principle while ensuring the design was easy to manufacture and test; as such, the design is not expected to be optimal in terms of miniaturization, mechanical performance, or sensing precision.The prototype has certain constraints on the design of the skin.The reflector ideally remains inside a ±35 • polar angle (i.e., angle from the vertical axis) range above the light angle sensor for a more linear relationship between the reflector's position and the photocurrents it generates.This geometric constraint helps define the shape of the interval cavity of the pillar and the aspect ratio of the X Y movement in relation to the light angle sensor.The reflector size was empirically chosen to achieve a good signal-to-noise ratio when in the neutral position and to prevent saturation of the light angle sensor from occurring for a significant Z deformation (≈25% of pillar height in its neutral position).Furthermore, the design conservatively uses two LEDs to more uniformly illuminate the cavity to ensure the intensity of reflected light reaching the light angle sensor is always sufficient, irrespective of the reflector position.While the shape of the internal cavity was chosen to be cylindrical with a hemispherical top to create an approximate circular symmetry in the association between X Y force magnitude force and X Y displacement magnitude, other internal cavity geometries are possible so long as the reflected light can reach the light angle sensor for all specified X Y Z displacements of the external pillar tip; the path from the reflector to light angle sensor cannot be occluded by the internal cavity walls.The minimum size of the pillar protrusion is determined by the footprint of the electronic components on the PCB.

D. Fabrication
The fabrication process is split into three steps: 1) silicone skin and pillar molding; 2) electronic circuit design and manufacture; and 3) top plate and cases/mounts, which are described in Sections II-D1-II-D3.
1) Silicone Skin and Pillar Molding: The pillar is molded from a two-part platinum cure silicone (Mold Star 20T) with a shore hardness of 20A.The top and bottom molds [Fig.4(a) and (b)] for the pillar were 3-D printed using stereolithography (Formlabs Form 2 printer, using Clear v4 resin), and then rinsed with isopropyl alcohol and post-cured at 60 • C for 20 min.The lower mold formed the external pillar shape and a void for the tracking dot to be added at the pillar's tip.The upper mold formed the internal cavity of the pillar and a void for the reflector to be added in a second step, shown in Fig. 4. The molding process is as follows: mixing the two silicone parts in a 1:1 ratio and adding 1:20 of black dye (Silc Pig) used to minimize the light reflection from the internal walls of the cavity.The silicone is then degassed before pouring into the lower mold; the upper mold is attached, and the silicone mix is left to cure [Fig.4(c)].
After demolding, a second batch of silicone was prepared using the same process, but, instead, a white dye (Silc Pig) was added.This silicone is carefully poured into the void formed at the bottom of the internal cavity of the pillar to form the reflector; the black silicone internal cavity is molded with a disk-shaped hollow (0.4-mm deep and diameter 1.5 mm) to receive the white silicone reflector.Similarly, the external calibration tracking dot is formed by pouring the white silicone into the void at the external pillar tip and leaving it to cure.Having the reflector and tracking dot made out of silicone forms a strong bond between them both and the black silicone skin, resulting in one piece of silicone for the LiVec skin.This is different from the PapillArray, which used a plastic reflector secured in silicone, having the potential to shift out of place [6].
2) Electronics: The PCB is populated with two 850-nm wavelength infrared LEDs (IN-S42CTQIR, Inolux Corporation) [37] and a light angle sensor (ADPD2140, Analog Devices) [35].The light angle sensor has a package size of 2 × 3 and 0.65 mm in height, with a radiant sensitive area of 0.31 mm 2 and has an approximately linear response within an angular field of view of 35 • [35].In addition, the light angle sensor has an integrated visible light-blocking filter to improve the rejection of other light artifacts.
The light angle sensor is interfaced with a photometric front end (ADPD1080, Analog Devices) with a 14-bit analog-todigital converter.The ADPD1080 was used to facilitate the readout of the data via inter-integrated circuit (I 2 C) communication and simultaneously control the LEDs.The LEDs are programed via the ADPD1080 to pulse at a high rate (every 4 µs with a duty cycle of 50%).LED pulsing reduces power consumption and heat generation from the LEDs and also enables excellent ambient light rejection.A low-dropout linear regulator (AZ1117C, Diodes Incorporated) generates a regulated 1.8 V to supply the ADPD1080.An IC-level shifter (NTS0304E, NXP Semiconductors) is used to take the 1.8 V of the general-purpose input/outputs (GPIOs) lines to 3.3 V [input level of the microcontroller (Teensy 4.1)] to enable readout from the ADPD1080 (via the Teensy 4.1 microcontroller) to a PC.
3) Top Plate and Cases/Mounts: The LiVec top plate was 3-D printed in white PLA plastic using a Prusa i3 MK3S+ printer (Prusa Research by Josef Prusa) to clamp the silicone skin (containing the sensing pillar) into the PCB using four M2 (2 mm) screws.This LiVec sensor assembly can then be attached to custom cases/mounts for use.

E. Experimental Protocol: Displacement and Force Calibration
1) Test Platform Setup: The test platform setup consists of a six degree-of-freedom (DoF) hexapod robot (H-820, Physik Instrumente, Germany), a transparent acrylic plate, a video camera (Logitech Streamcam, Logitech, Switzerland), a six-axis force/torque sensor (ATI Mini 40, ATI Industrial Automation, USA), and the LiVec sensor prototype (see Fig. 5).The force/torque sensor was mounted between a custom 3-D printed support mount and the LiVec sensor, and all were mounted on the hexapod.The ATI force sensor The data from the LiVec sensor, the ATI force data, and the hexapod position coordinates were uniformly resampled offline to match the sample times of the video recordings (i.e., all resampled to 60 Hz).
2) Protocol: The hexapod was programed to bring the pillar into contact with the acrylic plate until the desired compression (negative Z displacement) was achieved and then to move laterally (±X Y displacement) along one of two different trajectories.The Z displacement of 0 mm is defined when the pillar first makes contact with the acrylic plate detected by the sudden change in force seen by the ATI sensor.The two lateral movement trajectories (i.e., spiral and spoke patterns, see Fig. 5) were chosen to sample the 3-D deformation space of the pillar; each pattern was repeated at multiple pillar compressions (Z displacements of 0 to −2.2 mm in −0.1-mm increments).The spiral pattern was coded in polar coordinates with an angle that increased linearly with time from 0π to 10π radians (five revolutions) and a radius that increased linearly with time from a radius of 0.0 to 1.0 mm at a rate of 0.2 mm per revolution.The spoke pattern has 12 radial lines, each 30 • apart with a length of 1.0 mm.All movement patterns were executed at a speed of 2 mm/s.A training dataset was collected to calibrate the four light angle sensor raw photocurrent outputs to the independently measured 3-D displacement and force.A test dataset was collected to validate the calibration using the same protocol.
The video images of the pillar were used to track the actual X and Y positions of the pillar tip relative to the camera's frame of reference.The camera is calibrated in the plane of the acrylic plate.The actual X Y deformation of the pillar tip is obtained as the difference between the camera tracking of the pillar tip and the hexapod position, which accounts for any movement of the tracking dot on the pillar tip due to slip relative to the acrylic plate.The Z = 0 position is determined from the hexapod encoder position and the point of initial contact of the force signal, and this is used as the origin for the Z displacement.
3) Calibration of Displacement and Force: Multivariate polynomial regression, with a basic polynomial fit and equal weight to all inputs, was performed using the biased four raw light angle sensor photocurrent signals and the light intensity (sum of all four photocurrents) as the input, and the reference displacement (from camera and hexapod position) or force (from ATI force/torque sensor) as regression targets.We expect the ratios of the photocurrents (related to the X and Y angles of  signals and the light intensity are the independent variables in the regression model.To elaborate, there are six independent regression models, with three fourth-order models for estimating orthogonal X Y Z displacements and three third-order models for estimating orthogonal X Y Z forces; model orders were selected through ad hoc analysis of model residuals using the training set data, balancing improvement of model fit against model complexity.MATLAB was used to perform the regression using a MultiPolyRegress model.

F. Experimental Protocol: Vibration Testing
1) Test Platform Setup: A SmartShaker (K2004E01 smart shaker; The Modal Shop, Cincinnati, OH, USA) was used to vibrate the pillar tip along the Z -axis via a testing probe attached to the shaker and in contact with the pillar tip.The testing probe was brought into contact with the LiVec sensor pillar tip, so that the direction of the vibration was along the Z -axis.An oscilloscope provided the shaker with a set frequency sinusoidal input.A laser digital vibrometer (PDV-100 portable digital vibrometer; Polytec, Germany) was arranged to measure the testing probe displacement to verify the input vibration to the pillar tip.The arrangement of the test platform is shown in Fig. 6.The X -and Y -axes could not be evaluated, as it was infeasible to affix the vibration probe to the tip of the pillar to apply vibration in those directions, limiting our evaluation to only the Z -axis.
2) Testing Protocol: To test if the LiVec sensor could detect vibrations, the shaker was programed to deliver oscillations at a specific frequency, with each test lasting 10 s.The frequencies tested were 100-900 Hz in 100-Hz increments and 50 and 950 Hz.Multiple input vibration amplitudes were tested at two pillar Z compressions (approximately −0.65 and −1.6 mm).
The data from the LiVec sensor were captured at 1961 Hz using a Teensy 4.1 microcontroller.Data from the laser vibrometer (measuring velocity) were acquired at 10 kHz with a resolution of 0.02 µm/s.
3) Data Processing: We combine all four raw output channels as a proxy for the light intensity, which is then used to evaluate the response to vibrations.Fourier analysis was run on this intensity estimate from the LiVec sensor.In addition, trapezoidal quadrature integration was performed on the raw laser vibrometer data to convert from a voltage proportional to velocity (in units of mm • V −1 • s −1 ) to a voltage proportional to displacement (mm • V −1 ) to allow comparison between the reference laser measurement of the input displacement and the LiVec sensor estimate of light intensity, and then, a Fourier analysis was run.
A power spectral density (PSD) was estimated using a Welch periodogram (with a 0.5-s window, 25% overlap, and a Hamming window) of the LiVec intensity and input vibration to quantify the frequency response of the LiVec sensor.The transfer function magnitude for the LiVec response across the whole frequency range was calculated using the ratio of the LiVec intensity estimate amplitude and the input displacement amplitude calculated from the laser measurement.This was done for the three sinusoidal amplitudes used to drive the SmartShaker vibrator.
G. Experimental Protocol: Temperature Testing 1) Test Platform Setup: A Prusa i3 MK3S+ heated 3-D printer bed was used as a hot plate to provide a controlled heat source to the tip of the LiVec sensor.The LiVec sensor was inverted and mounted above the printer bed using a clamp and stand.A thermocouple was placed in contact with the base of the LiVec silicone skin pillar protrusion to measure the actual temperature of the LiVec sensor skin.
2) Test Protocol: To test how the LiVec sensor responds to changes in temperature, the heat source was programed to increase its temperature until the silicone measured 60 • C. The temperature range tested was from room temperature (approximately 20 • C) to a maximum of 60 • C; between tests, the entire apparatus, including the LiVec sensor, was cooled back down to room temperature.Each test was repeated three times.The raw photodiode channels, along with the X Y Z displacement and X Y Z force estimates output by the previously calibrated regression equations, were continuously recorded from the LiVec sensor as the temperature increased.Two tests were performed to evaluate the LiVec sensor response to temperature.
Test 1: The LiVec sensor was inverted, and the sensor tip was positioned 2 mm above the heat source to simulate changes in the ambient environment.
Test 2: The LiVec sensor was inverted and brought into contact with the heat source and subjected to approximately 0.5-mm compression in the Z -direction to simulate contact with objects with different temperatures.

A. 3-D Displacement and 3-D Force
An illustrative recording of the four raw outputs channels of the light angle sensor with respect to time is shown in Fig. 7(a) for an X Y spiral movement of starting radius 0.1 mm and increasing linearly with time to a final radius of 1 mm, Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.7(b) and (d)].Similarly, the estimated force from the multivariate regression model also tracks the actual force well [see Fig. 7(c) and (e)].Pillar slips against the acrylic plate are apparent and manifest as deviation from the hexapod spiral pattern [shown in Fig. 7(d) and (e)].Still, the regression model accounts for this, since the regression target used is the difference between the hexapod X Y position and the X Y position of the camera-tracked tracking dot at the top of the LiVec sensor pillar.

TABLE I DISPLACEMENT ERROR TABLE: THE OVERALL BIAS (MEAN ERROR), PRECISION (SD), AND MAXIMUM (TEST DATASET) 3-D DISPLACEMENT
The total number of data samples used to train the multivariate regression model from the spiral and spoke training patterns (see Section II-E2) was 238 881.The multivariate regression model was then tested on a dataset created using the same protocol, containing 240 063 data samples.The 3-D displacement and 3-D force estimate errors statistics for the test dataset are summarized in Tables I and II.The precision of the measurements estimates is taken as the standard deviation (SD) of the error, and the bias of the measurement estimates is taken as the mean difference between the measurement estimate and its true value.
The average displacement error is 2.13 µm, across all three axes.The X Y Z displacement estimate error precision is in the order of tens of µm.The average error for the force is −15.0 mN, across all three axes The X Y Z force estimate error precision is in the order of tens of mN.
This LiVec sensor prototype has a tested displacement range of approximately 2.0 mm in the X -and Y -directions and −2.2 mm in the Z -direction.The maximum X Y Z displacement percentage errors are 5.2%, 4.7%, and 0.3%, respectively.Similarly, the dynamic force range is ±2 N in X and Y and −6 N in Z .The maximum X Y Z force percentage errors are 3.5%, 2.9%, and 1.5%, respectively.
To visualize the distribution of the estimated error across the whole test dataset and to estimate the dependence of the error on the range of force or displacement applied, violin plots (Fig. 8), scatter, and Bland-Altman plots (Figs. 9 and 10) are shown.These plots help visualize the multivariate regression model bias and to identify anomalies, biases, or skew in the estimated displacements and forces.The violin plots show that the error in the estimates is all centered around zero.The displacement biases are −10.8,15.4, and −1.8 µm for X Y Z , respectively, and the difference between the mean and median is 5, 10, and 2 µm for X Y Z , respectively, with the median being 5 µm closer to zero for each estimate error.Similarly, for force, the biases are −8.5, −8.5, and 28.0 mN for X Y Z , respectively, and the difference between the mean and the median is 0, 1.4, and 12 mN for X Y Z , respectively.
The displacement estimates have a Pearson r -squared value for X Y Z of 0.98, 0.97, and 1.0, respectively.Similarly, the

B. Vibration Results
As the vibration analysis was performed using the raw channels from the ADPD2140, the signals from the LiVec sensor and the laser were scaled to arbitrary units in all plots for visualization purposes.The shaker probe was brought into contact with the LiVec sensor pillar tip with two different compressions (in different experiments) to explore how sensitivity to vibration varies with this compression level.A typical recording of the normalized (unity-based normalization to arbitrary units) LiVec sensor intensity (defined in Section II-F3) and the normalized measured vibration input with respect to time shown for a 400-Hz vibration of the shaker applied to the LiVec sensor at two different compressions are shown in Fig. 11(a) and (b).The sampled sinusoidal vibration waveform measured by the LiVec sensor captures the shape and frequency of the input vibration as measured by the laser.The behavior is similar at the two tested compressions (Fig. 11).From inspection, the LiVec sensor intensity signal corresponds well (relative amplitude and sinusoidal shape) with the laser-measured vibration displacement across all the frequencies and sensor compressions tested; this correspondence is quantified more rigorously later in the article using Fourier analysis.for both compression levels for the LiVec intensity signal and the measured input vibration, thus confirming that the LiVec sensor is sensing the input vibration frequency.
Fig. 13(a) and (c) shows the transfer function magnitude for the LiVec sensor across the range 50-950 Hz for both compression levels.The transfer function magnitude was calculated as the gain of the LiVec sensor, i.e., the ratio of the maximum of the PSD amplitude of the LiVec sensor intensity signal divided by the maximum of the PSD amplitude of the laser-measured vibration displacement signal.This shows that the sensor is sensitive to vibration across the 50-950-Hz range tested, with the LiVec intensity estimate remaining largely above the noise floor of the LiVec sensor (minimum of 20 dB) for the entire frequency range [Fig.13(b) and (d)].We can also observe that when the input vibration is too small (50-mV sinusoidal signal from the signal generator driving the shaker motor, at frequencies greater than 500 Hz), the sensor does not detect the vibration; i.e., the magnitude of the transfer function falls into the noise floor of the LiVec sensor [Fig.13(b) and (d)].From inspection, the sensor shows good sensitivity at both compressions, confirming the sensor's sensitivity to vibrations.

C. Temperature Results
Fig. 14 shows the four raw outputs channels of the light angle sensor with increasing temperature from 20 • C. (room temperature) to 60 • C. In Test 1, when the sensor is uncompressed, the raw light angle sensor outputs decrease with increasing temperature [Fig.1(a)].In Test 2, when the sensor is in a compressed position [Fig.14(b)], the raw light angle sensor outputs increase with changing temperature up to 45 • C before decreasing again slightly.These changes in the raw output affect the calibration for 3-D force and 3-D displacement estimates, causing a temperature drift in these estimates, particularly the Z -axis, shown in Fig. 14(a) and (b).
To compensate for the effect of temperature on the raw light angle sensor channel values used to estimate force and displacement, we explored using a regular rebiasing of the LiVec sensor.To get a bias value, the average of the sum of the four raw light angle sensor photocurrents was recorded for 10.5 s.This value was then used to normalize the light angle sensor photocurrents and intensity before they are input to the regression model.A rebiasing was performed at every 1 • C temperature increase and applied to 10 s of sensor data around that temperature.After rebiasing the sensor, the displacement and force estimates showed minimal deviation for from the expected zero displacement/force value [Fig.15(a) and (b)], which is when that with a smaller sinusoidal input displacement, it is only responsive up to 500 Hz, because, after this, the liVec intensity estimate was in the noise floor, meaning a vibration could not be identified.The larger input amplitudes could only be tested at higher frequencies (around 200-Hz upward) due to the testing probe not remaining in contact with the pillar at lower frequencies.
the sensor is not in contact/resting position.After rebiasing, as the temperature increases, the Z -axis for both force and displacement is affected more than the X -and Y -axes.

IV. DISCUSSION
This article introduces a new small-scale optical sensing concept (called LiVec) that uses a light angle sensor to transduce the direction and intensity of light incident on the detector and allow the measurement of the displacement and/or force applied (including vibration) to the tip of the sensor.
Using a prototype proof-of-concept design, we show that this sensing approach can detect 3-D displacement and 3-D force with similar measurement estimate errors to current state-of-the-art sensors (see Tables I and II) and additionally can detect vibrations from 10 to 950 Hz with an average transfer function magnitude of 60 dB and a minimum of 10 dB above the noise floor.The LiVec sensor has a displacement measurement range of ±1 mm in the X -and Y -axes and −2.2 mm in the Z -axis and a force measurement range of ±1 N in the X -and Y -axes and −6 N in the Z -axis.Taking the precision of the measurement estimates as the SD of the error, the displacement measurement precision of the LiVec sensor is 52 µm in X , 67 µm in Y , and 18 µm in Z .The force measurement precision of the LiVec sensor is 48 mN in X , 49 mN in Y , and 93 mN in Z .Furthermore, the displacement bias, defined as the mean difference between the measurement estimate and its true value, of the LiVec sensor is −10.8 µm in X , 15.4 µm in Y , and 1.8 µm in Z .The force bias of  the LiVec sensor is −8.5 mN in X and Y and −28 mN in Z .The results show good linearity (average R-squared value across all axes is 0.99).The percentage errors were less than 4% of the displacement and force dynamic ranges, respectively.The LiVec sensor was also responsive to mechanical vibrations in the range of 10-950 Hz, validated using a laser vibrometer, covering the frequency range of human mechanoreceptors [38].Taken together, the measurement bias and precision and excellent bandwidth of the LiVec sensing principle show great promise for use in robotic manipulation and grip control applications.The LiVec sensor can be used in real time, as shown in the real-time demonstration provided in the Supplementary Material.
The LiVec sensor has shown to be sensitive to environmental temperature changes and object temperature changes.We hypothesize that as the temperature increases, the silicone expands, moving the reflector further away from the neutral position.Furthermore, the reflectance coefficient of the silicone will decrease with increasing temperature [39].These two effects may explain the decreases in the raw light angle sensor outputs observed in Test 1 [Fig.14(a)].Similarly, the increase in the raw channel intensities in Test 2 [Fig.14(b)] may be caused by the silicone around the reflector expanding inward, as the outward expansion of the silicone is constrained due to contact with the heat source, causing an increase in the amount of reflected light due to the reflector surface moving closer to the light angle sensor; all the while the reflectance coefficient of the silicone is expected to decrease with increasing temperature, which may explain the decrease in received light above 45 • C.However, as we cannot measure the pillar tip Z position at each temperature with the precision required, nor the exact position of the reflector hidden inside the sensing cavity, we cannot disentangle silicone expansion effects from silicone reflectance changes or potential circuit heating effects (LED intensity changes, amplifier gains changes, and so on).However, regular rebiasing may be able to solve this issue, allowing the sensor calibration to remain valid at other temperatures or alternatively, using an onboard temperature sensor may allow for active compensation of temperature-related drift.

A. Design Advantages
The LiVec sensor has a displacement measurement range of ±1 mm in the X -and Y -axes and −2.2 mm in the Z -axis and a force measurement range of ±1 N in the X -and Y -axes and −6 N in the Z -axis.The LiVec sensor has a measurement precision of −10.8 µm in X , 15.4 µm in Y , and 1.8 µm in Z displacements; and −8.5 mN in X and Y forces and −28.0 mN in Z force, and matches that of current state-of-theart sensors (see Tables I and II).The GelSight sensors family has a measurement accuracy of <50 mN [40], while the TacTip technology family has achieved 3-D surface localization of 0.1 mm/0.3 • [16].The PapillArray has a displacement error of tens of mm and a force error of tens of mN [27].The magnetic sensor uSkin uses a Huber regression for calibration with R-squared results of upward of 0.97 [13]; however, precision values are not reported.The anisotropic waveguide optical micro/nanofiber tactile sensor skin has an average accuracy of 28 mN for a sensing range of 1 N [25].The commercially available optical force/torque sensor [OnRobot, HEX-E/H QC (previously Optoforce)] has a noise-free resolution of 0.2 N in X Y and 0.8 N in Z [23], [41]; however, this is not a tactile sensor, in the sense that it is a rigid force/torque sensing technology typically mounted between robotic components, such as between a gripper and the wrist of a robotic arm.Thus, the LiVec sensor has a measurement precision comparable to the existing tactile sensors but has considerable improvements in manufacturability and significant potential for further miniaturization.
The LiVec sensor is robust to interference from external light sources.Passive strategies used to reduce the impact of external pollution include using black silicone to reduce the amount of external light reaching the internal cavity and the light angle sensor.Furthermore, the light angle sensor has an integrated optical filter that rejects ambient visible light.In addition, the photometric front end (ADPD1080, Analog devices) performs active rejection of ambient light by pulsing the LEDs (2-µs pulses), bandpass filtering the resulting photocurrents to reject ambient light.
The durability of the sensor is improved by having a fully soft, compliant silicone pillar with an embedded reflector made of the same material, unlike the PapillArray tactile sensor [6].
The LiVec optical transduction method has some considerable advantages over other tactile transduction methods.For instance, it is robust against electromagnetic interference.This may prove to be superior to magnetic-based sensors, such as the uSkin [12], [13], [14], which is prone to interference from ambient magnetic fields and the influence of nearby ferromagnetic materials, which have the capability to perturb the field generated by the permanent magnetic embedded each tactile sensing element.Such transduction methods that use magnetic fields that do not actively change their strength or direction (relative to the permanent magnet) are inherently passive and cannot perform active artifact rejection.
In addition, the LiVec sensor's optical transduction method endows it with excellent sensitivity to mechanical vibrations; the primary limiting factor in the LiVec's vibration sensitivity is the stiffness of the silicone that must transfer displacement from the surface of the sensor to the internal reflector, but this stiffness is configurable, as it is determined by the shore hardness of the silicone, which can be altered.The LiVec optical transduction method has a large bandwidth of at least 950 Hz, which is not achieved by many tactile sensors [2].Looking to human touch, which for many sets the standard for tactile sensor design, human mechanoreceptors can detect skin vibrations in the range of 20-700 Hz [38]; the LiVec sensor spans this frequency range.The BioTac sensor response to vibrations outperformed human sensitivity in experimental tests; however, this system is limited to a single channel and uses a dedicated hydrophone to transduce vibration [42].Moreover, all sensors using a camera imaging transduction method, including GelSight/GelSlim [8], [9], [29], [32] and the TacTip [16], [20], currently suffer from low frame rates (30-90 FPS [16]), meaning high-frequency vibrations cannot yet be sensed.
Another drawback of camera-based sensors is the high-computational requirements to process the data needed.One method to reduce the processing required is to use event-driven cameras.This was proposed by Taunyazov et al. [43] using a neural network and an event-based camera with good success compared to other tactile deep learning methods.While using an event-driven camera can reduce the amount of computational power required, as they only respond to changes above a detection threshold, very low-frequency changes may be missed, which affects their ability to sense static forces/displacements. Therefore, our proposed LiVec sensor has these advantages over camera-based sensors, as it has a simple mapping between the sensor output and force/displacement and only four output signals rather than 10 000s of camera pixels.
1) Height Comparison of Similar Tactile Sensors: The motivation for using the ADPD2140 light angle sensor was to reduce the thickness of the sensor, such that it can more seamlessly be integrated into robotic and/or prosthetic grippers.This proof-of-concept prototype, with a total height of only 11 mm, is already extremely competitive with state-ofthe-art sensors.Traditionally, optical-based tactile sensors have been relatively large and bulky with significant height, making it difficult or at least challenging to integrate them into robotic grippers (see Table III).This is especially the case for camera-based optical sensors, for which reducing the height of the sensor is restricted by the need to place the camera a focal length from the surface/skin of the sensor.The literature identifies this as an issue, and efforts continue toward making camera-based tactile sensors more compact.The latest compact design in the GelSight family is the GelSlim 3.0, which uses a shaping lens and a Raspberry Pi wide-angle camera module to achieve a smaller size.Still, the cost of constraining the design space results in a reduced sensing field due to the spherical distortion induced by the wide-angle lens [9].A different camera-based design, the OmniTact, has a much larger sensing field, with sensing on all sides of the finger-shaped sensor, but the image resolution is worse [15].In the TacTip family of sensors, the most compact is the TacTip GR2, built for a GR2 gripper, with a reduced overall height of 44 mm [16], [20].The sensing principle of the PapillArray sensor [6], [27] uses a camera obscura, which requires two PCBs.This poses challenges for miniaturization and for ensuring the reproducibility of the sensor component geometries and relative positions.In turn, these factors influence the calibration characteristics of the device [27].In the first published embodiment, the silicone pillar had a height of 20 mm, not including the PCBs.The more recent version of the PapillArray sensor (Development Kit v2.0 [44]) developed by Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.
the Contactile company has a pillar height of 4.2 mm (above the casing) and a casing height of 8.6 mm, making an overall sensor height of 12.8 mm [44].Unlike camera-based tactile sensors and the Papillarray sensor [27], our LiVec sensor has fewer apparent barriers to miniaturization.It is already smaller than current camera-based sensors, albeit if only sensing 3-D force and displacement at a single point.
The successful magnetic triaxial uSkin tactile sensors [14] have a height of 6.05 mm, with a spatial density of 4.7 mm between individual elements.This is smaller than the LiVec sensor and half the height.But, this sensor uses magnetic transduction methods and is, therefore, prone to electromagnetic interference, as discussed earlier.Jiang et al. [31] used an optical microfiber to create a flexible slip sensor with a rough height of 2 mm, but its tactile information is limited to surface information.In contrast, Zhang et al. proposed a miniature thin optical vision-based tactile sensor based on a bionic compound eye structure.The sensor has a height of 5 mm and has 3-D force sensing with contact geometry perception [30].This is one of the first tactile sensors at this scale, which retains the advantages inherited from vision-based tactile sensors.However, increasing or changing the shape and size of the sensing area will be challenging, as it uses CMOS imaging.Indeed, using a CMOS array restricts the sensor's base to be flat, as the individual sensing elements are not discrete.Distributed sensing approaches, such as uSkin, which have discrete sensing elements, could be spatially distributed on flexible PCB to cover large curved surfaces [14].
An active research area in the tactile sensing field is that of electronic skins.Electronic skins are designed to be very thin, cover large areas, and conform to curved surfaces.Electronic skins are typically capable of performing pressure sensing [46], [47], [48].While electronic skins have many advantages of thin size, easy integration, and flexibility, they only transduce one property, such as pressure or shear forces [47].This could be a significant weakness concerning their use in robotic manipulation, where three-axis force and/or torque sensing can be extremely beneficial.Moreover, to achieve such small-thickness high-density arrays, specialized and expensive manufacturing methods are often required, which has limited the widespread adoption of such technologies.
Compared with current state-of-the-art sensors, the LiVec sensor demonstrates a small height with maintained sensing capability.The reduced height allows it the potential to be seamlessly integrated into the existing robotic grippers without limiting the range of movement of the gripper fingers or significantly increasing their size.Moreover, it has the potential to be instrumented on curved surfaces.

B. Design Limitations
It is important to note some limitations of the LiVec sensor design.The most noteworthy limitation is the occurrence of light saturation resulting in erroneous estimates of displacement and/or force.Light saturation can occur if there is too much compression on the pillar or the LEDs are too bright.Indeed, if the silicone is compliant enough or the force large enough, the reflector can come very close to the light angle sensor and possibly even touch it.For this to occur in the prototype described here, the pillar must be compressed with a normal force larger than 6 N.However, this upper force limit can be configured by changing the stiffness of the silicone.However, saturation is easily detectable, as the analogue digital converter (ADC) will reach the top of its range, so that these measurements can be flagged as inaccurate.
Another limitation of the LiVec sensor is that the soft skin needs to be securely attached to the hard PCB.To do this currently, it requires four screws and a top plate.However, the screws take up a large amount of space, which could, otherwise, be used to further miniaturize the overall design by improving the PCB routing density and electronic component placement density.This issue of the fixation of the skin to the PCB will be an area for future improvement when attempting to densely arrange the LiVec sensing elements as part of a larger tactile sensing array.Moreover, ensuring a good signalto-noise ratio of the light angle sensor photocurrents led us to choose a conservative design that used two LEDs to ensure a sufficiently large light intensity reached the light angle sensor detection area, irrespective of the reflector position when the pillar is deformed.However, using two LEDs increased the sensor's footprint and the overall pillar size; future work will explore the possibility of using a single LED and possibly increasing the reflector size to compensate, which should ultimately reduce the footprint of the LiVec design and allow for more dense spatial arrangements when arrayed.
The sensor's sensitivity to temperature is a limitation.Indeed, the output photocurrents of the light angle sensor change with the temperature, which can cause incorrect displacement measurements by a maximum of 0.9 mm and incorrect force measurements of a max of 0.5 N at 60 • C, shown in Fig. 15.Such drift with temperature may be caused by multitude of independent effects, such as the various circuit heating effects (including amplifier gain drift and LED output intensity changes), silicone expansion, and silicone reflectance changes.In the limited investigation performed in this article, regular rebiasing when the sensor is mechanically unloaded shows promise as a method to compensate for the compound effect of all sources of temperature-related drift.However, comprehensive validation of this method for all possible X Y Z displacements of the sensor would require repeated calibration experiments in a temperature-controlled environment, which may damage our robotic and measurement apparatus by exposing them to these temperatures; thus, this testing is currently beyond our capabilities.
Temperature-induced output drift in tactile sensors is a widely acknowledged challenge in tactile sensor design.Adding a temperature sensor to the design is a potential solution to correct the sensor drift [49].However, including such sensing may limit the achievable spatial density of the individual tactile sensing elements and still requires a solution to improve the measurement drift with temperature change [50].A common solution to compensate for temperature drift is to use a rebiasing procedure before sensor use; this method is used in commercially available products, such as the ATI Industrial Automation force/torque sensors [51], [52].More recently, electronic skins have implemented temperature compensation systems, such as regular biasing [49], [53].In addition, knowing the temperature is a beneficial tactile property adding to a sensor's utility, so adding a temperature sensor may serve a dual purpose.
Another limitation of our temperature testing was our limited experimental setup, whereby extreme temperatures (particularly cold) could not be created.Moreover, this setup did not allow us to deform the pillar laterally at various temperatures.Our future work will involve further temperature testing, including further explorations of the rebiasing and potentially embedding a temperature sensor onto the LiVec PCB to accurately compensate for changing temperatures.In addition, we will explore using two light angle sensors to eliminate the reliance on LED output intensity in the transduction principle to create a more robust design.
The design of the calibration experiments and the reflector placement creates a limitation in the regression models.The regression models attempt to infer 3-D force/displacement at/of the tracking dot using light reflected from the diffuse reflector to the light angle sensor, which, of course, is dependent on the reflector's position, orientation, and shape.There is no unique one-to-one relationship between the deformation of the outer surface of the silicone pillar and the position, orientation, and shape of the reflector embedded within it.As the calibration protocol kept the acrylic plate parallel to the sensor PCB (see Fig. 5), all data used to train and test the regression models are geometrically constrained in this way.Therefore, the measurement precision may worsen if the pillar is contacted by an object/surface arriving at some other angle or by a nonplanar surface.This issue could be mitigated by moving the reflector closer to the pillar's external tip and possibly making it smaller, but with a reduction in reflector size, causing a reduction in signal-to-noise ratio.It could also be possible to replace the reflector with a small LED near the pillar's tip to improve this issue, but with a significant increase in complexity associated with embedding electronics in the silicone skin.
The LiVec sensor design requires a calibration procedure to measure the 3-D displacements and 3-D forces, which can be time-consuming and required for every new design.Future work will explore the use of generalized regression models that obviate the need for calibration of every sensing pillar manufactured.

C. Design Optimization
The LiVec sensor presented here is a proof-of-concept design, which means there are parameters in the LiVec design, which could be optimized to improve performance further.The size and shape of the skin can be customized for a specific application if the interval cavity is maintained so as not to alter the sensing principle.However, alterations of the skin compliance or geometry will affect the sensing range and sensitivity.Future work will explore using various shapes for the outer surface of the pillar protrusion to learn what parameters allow certain specifications to be met.
Furthermore, the robustness of the LiVec sensor to temperature variations deserves further investigation.Changing temperatures not only affect the silicone skin, causing expansion but can also induce drift in amplifier gains and LED output intensities.LED pulsing is already utilized to reduce the intrinsic heating effects of the LEDs when their output light is absorbed by the silicone pillar.However, changes in ambient temperature are not yet compensated for.In our preliminary investigations of this issue, biasing the photocurrent values when the sensor pillar is unloaded/undeformed appears to compensate for the broad range of temperatures tested.However, comprehensively validating the X Y Z precision of the calibrated sensor after rebiasing at multiple temperatures is not feasible with our current experimental apparatus.Another avenue to explore is the addition of a temperature sensor to the LiVec sensor to enable temperature compensation without the need for unloaded/undeformed rebiasing.

D. Potential Applications for the LiVec Sensor
Providing robotic fingers with tactile sensation mimicking human touch may give them the ability to perform dexterous manipulation of an object with a secure grasp.The goal of the LiVec sensor presented here was to develop a proof-ofconcept transduction method with good potential to be arrayed into a larger tactile sensing system, such as a fingertip sensor for a robotic gripper.By having an array of LiVec sensing elements, the localized and precise displacements measurements allow the identification of incipient slip occurrence and subsequent response to manage the security of the grasp, say by increasing the grip force.For example, Osborn et al. showed how textile force sensors with a precision of 0.01 N could provide tactile feedback to prosthetics by enabling contact and slip detection in a grasping task [54].This level of precision allowed for the sensitivity to detect perturbation due to object slip; since the LiVec sensor has a similar precision and can detect displacement means, it shows promise to be applicable for prosthetics.Being able to sense vibrations allows the LiVec sensor to better approximate the function of human mechanoreceptors, which individually sense vibration, pressure, and stretch [55].Vibrations allow humans to perceive textures from their fingertips [42].The BioTac sensor has shown texture discrimination using vibrations in the range of 250 Hz [42]; since the LiVec sensor covers this range, it can potentially be used for texture discrimination, as achieved by Jimenez and Fishel [56], showing how vibrations could be used for texture discrimination in prosthetics.Given measurement precision of the LiVec sensor and its potential to be arranged as a larger tactile sensing array, it is designed to be used for improving dexterous manipulation, primarily in the role of grasp stability by incipient slip detection, but it also shows promise for improving prosthetics tactile sensation.

V. CONCLUSION AND FUTURE WORK
In this work, we proposed a small compliant tactile sensing element with a novel instrumentation approach for measuring the 3-D displacement, 3-D force, and vibration experienced by the pillar tip.The new design uses only one PCB, simplifying the overall sensor design and manufacturing process relative to the related PapillArray design [6], and promises to enable substantial volume (in particular sensor height) reduction in Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.
future designs.The small height of the design means that the pillar can be easily implemented onto commercially available robotic grippers and additionally implemented as a modular part in a larger tactile array, with the potential to measure grip and load forces, incipient slip, and friction.
The LiVec sensing principle could have many potential applications in tactile sensing and robotic manipulation.The primary motivation for developing this new sensing principle was to improve robotic manipulation by providing tactile sensing for grip control in a compact form factor that can be easily integrated onto robotic grippers without making the gripper fingers large and unwieldy.However, the LiVec sensor may find other future uses in minimally invasive robotic surgery control, rehabilitation and prosthetics, and teleoperation systems.
In the future, we will continue to explore the miniaturization potential of the LiVec sensing principle.The use of sensor arrays and skins can be used for slip detection and subsequent grip force control during robotic manipulation tasks.Therefore, in our future work, we aim to make an array of LiVec sensing elements and deploy them to achieve improved grip force control during robotic manipulation.The simple design of the pillar using minimal sensing components and a simple manufacturing process allows for the LiVec sensor to be modular and designed for a specific application with ease.Promisingly, using flexible PCBs, the LiVec design could move away from a rigid PCB backplane to instrument curved surfaces for improved sensing capabilities to cover robotic end effectors and limbs.The advances in miniaturization of precise tactile sensing will significantly improve dexterous robotic manipulation.

A
Tactile Sensing Concept for 3-D Displacement and 3-D Force Measurement Using Light Angle and Intensity Sensing I. INTRODUCTION

Fig. 1 .
Fig. 1.Sensing principle of the LiVec sensor.(a) 3-D cutaway illustration of the sensor showing the main components.(b) Pillar in its undeformed neutral position showing a subset of possible light paths from the LEDs illuminating the pillar cavity.The light, which reaches the pillar cavity walls, is mostly absorbed by the black silicone.Light reaching the reflector is diffusely reflected, with some of that scattered light arriving at the detection area of the ADPD2140 light angle sensor.Note that, it is not trivial to illustrate that light reflecting from all parts of the disk-shaped reflector is incident on all parts of the light angle sensor detector area, but the average incident angle of the light when the sensor is in this neutral position is 0 • (relative to the normal to the PCB); again, in this illustration, only some light rays are shown for clarity.(c) Illustration of a lateral displacement of the external pillar tip in the X-direction by deforming the dome-shaped pillar.The average incident angle and intensity of photons arriving at the ADPD2140 light angle sensor change with the reflector position.(d) Illustration of Z compression with no lateral displacement.The reflector is nearer to the ADPD2140 light angle sensor, and so, the average incident light intensity on the light angle sensor is increased.Therefore, intensity changes can be used to help infer Z displacement.

Fig. 2 .
Fig. 2. (a) Simulation representation of the light propagation inside the cavity of a pillar deformed −1.5 mm in the Z-direction and +1 mm in the X-direction.(b) Difference in intensity between using one or twoLEDs when deforming the pillar 1 mm in the negative versus positive X-direction (X− versus X+, respectively), suggesting the two LEDs would be preferable to achieve adequate signal-to-noise ratio in the proof-of-concept prototype planned for this article.

Fig. 3 .
Fig. 3. (a) Photograph of the LiVec sensor prototype.In the prototype pictured here, the external tip pillar contains a white tracking dot for XY displacement calibration (based on external video tracking), four screws, and a white top plate that clamps the silicone skin onto the sensing PCB beneath.The LiVec sensor is mounted on a black stand for this picture.(b) Cross-sectional drawing of the LiVec sensor, showing the components: PCB, light angle sensor (ADPD2140, Analog Devices), two infrared LEDs, black silicone skin containing a pillar protrusion with an internal hollow cavity, white silicone reflector, and a top plate used to hold the parts together.The LiVec sensor pillar has a height of 9.0 mm above the top plate and an external diameter of 8.5 mm.The top plate is 0.75 mm above the top of the PCB holding the skin in place.The height between the light angle sensor and the reflector is approximately 5.5 mm, and the top of the cavity skin is 2.5-mm thick to encompass the reflector and tracking dot.The LiVec sensor has a height of 11 mm, from the backside of the PCB components to the hemispherical tip of the silicone pillar.The LiVec reference frame is shown, illustrating that the X and Y vectors are in the plane of the PCB and the Z vector is out of the plane of the PCB.

Fig. 4 .
Fig. 4. Fabrication and assembly of the LiVec sensor silicone pillar.(a) Lower mold for the pillar.(b) Upper mold for the pillar forms the pillar cavity.(c) Attached upper and lower molds during curing of the black silicone forming the pillar.(d) Top plate.(e) Final silicone skin.(f) Fabricated silicone pillar held by the top plate onto the sensing PCB and mounted onto a stand.has a resolution of +/−20 N in X and Y and 60 N in Z .The force data were sampled at 200 Hz.The camera for the X Y calibration was mounted approximately 310 mm above the LiVec sensor and used to image the external tracking dot on the pillar tip.The LiVec sensor data were sampled at 500 Hz, and the video was captured at 60 FPS with a resolution of 1280 × 720 pixels and a camera calibration error of 0.15 pixels.The hexapod position coordinate data were recorded and used to synchronize all the collected signals.The data from the LiVec sensor, the ATI force data, and the hexapod position coordinates were uniformly resampled offline to match the sample times of the video recordings (i.e., all resampled to 60 Hz).2) Protocol: The hexapod was programed to bring the pillar into contact with the acrylic plate until the desired compression (negative Z displacement) was achieved and then to move laterally (±X Y displacement) along one of two different trajectories.The Z displacement of 0 mm is defined when the pillar first makes contact with the acrylic plate detected by the sudden change in force seen by the ATI sensor.The two lateral movement trajectories (i.e., spiral and spoke patterns, see Fig.5) were chosen to sample the 3-D deformation space of the pillar; each pattern was repeated at multiple pillar compressions (Z displacements of 0 to −2.2 mm in −0.1-mm increments).The spiral pattern was coded in polar coordinates with an angle that increased linearly with time from 0π to 10π radians (five revolutions) and a radius that increased linearly with time from a radius of 0.0 to 1.0 mm at a rate of 0.2 mm per revolution.The spoke pattern has 12 radial lines, each 30 • apart with a length of 1.0 mm.All movement patterns were executed at a speed of 2 mm/s.A training dataset was collected to calibrate the four light angle sensor raw photocurrent outputs to the independently measured 3-D displacement and force.A test dataset was collected to validate the calibration using the same protocol.The video images of the pillar were used to track the actual X and Y positions of the pillar tip relative to the camera's

Fig. 5 .
Fig. 5. Robotic test platform for displacement and force calibration and the XY hexapod robot stage movement patterns for displacement and force mapping.(a) Picture showing the test rig setup, which includes the camera mounted above the hexapod robotic stage as an independent reference of the XY displacement of the pillar tip tracking dot, should it slip against the acrylic plate.The LiVec sensor and ATI force sensor are mounted on the hexapod robotic stage.The acrylic plate is held by a T-slot frame between the camera and the LiVec sensor.(b) Diagram of the test platform showing the placement of the equipment in relation to the hexapod robotic stage.(c) Spiral pattern.(d) Spoke pattern.The spiral and spoke patterns were each repeated in steps of −0.1-mm Z compression to a maximum of −2.2 mm.The spiral starts at a radius of 0.0 mm and increases linearly in radius to a maximum of 1.0 mm after five revolutions.The spoke has a 1.0-mm radius and 12 spokes equally spaced at 30 • intervals.
the incident light) to remain unchanged, given a change in LED intensity.The raw photocurrents are biased by taking an average intensity across 200 samples at the beginning of a test when the pillar is in an undeformed neutral position.The raw photocurrent signals and light intensity are then divided by the bias value.The biased raw light angle sensor photocurrent

Fig. 6 .
Fig. 6.Test platform for the vibration experiment.(a) Diagram of the vibration testing setup showing the SmartShaker, LiVec sensor, testing probe, and laser vibrometer.(b) Top view of the test setup.(c) Image of the testing probe and the LiVec sensor to show how the vibration was transferred from the shaker shaft, via the test probe, to the LiVec sensor pillar tip.

Fig. 7 .
Fig. 7. Displacement and force mapping for a spiral pattern at −1.3-mm Z compression.(a) Four raw output channels from the ADPD2140 light angle sensor for a spiral movement of a 1-mm outer radius are plotted as each sample (Y-axis).The ADPD2140 channels are categorized as X1 (blue), X2 (red), Y1 (yellow), and Y2 (purple), representing their link to the X and Y angles that the light makes with the light angle sensor detector area.There is a 14-bit ADC in the ADPD1080 with an offset of 0 × 2000.(b) Actual (solid) and estimated (dashed) XYZ displacement.(c) Actual (solid) and estimated (dashed) XYZ force.(d) and (e) Actual and estimated XY displacement and force, respectively, for this illustrative spiral pattern.The spiral shape is not perfect, as the pillar slipped, but the video tracking accounts for this when generating the XY displacement values passed to the regression model during training.

Fig. 8 .
Fig. 8. XYZ displacement and force estimate error distributions, visualized in violin plots.(a) Displacement estimate error.(b) Force estimate error.force estimates have a Pearson r -squared value for X Y Z of 0.95, 0.98, and 1.0, respectively.The X and Z displacement Bland-Altman results [Fig.9(b) and (f)] show a small bias (mean) and no qualitative evidence of proportional bias in the error values.However, the Y displacement Bland-Altman results [Fig.9(d)] show a slight overestimation across the data (as the interquartile range (IQR) extending a further 0.02 mm into the positive Y ) but with proportional bias across the range of displacements.Overall, the displacement Bland-Altman results confirm the reliability of the displacement regression model with SD of the differences of 52, 66, and 17 µm for X Y Z , respectively.The X Y Z Bland-Altman result for the force estimates [Fig.10(b), (d), and (f)] shows no qualitative evidence of proportional bias, confirming the reliability of the force regression model, with the SD of the differences being 48, 49, and 93 mN for X Y Z , respectively.

Fig. 12 (
a) and (b) shows the normalized PSD of the normalized signal for the LiVec sensor intensity estimate signal and the measured vibration input displacement signal measured with the laser for a 400-Hz vibration of the sensor at both compression levels.A significant peak can be seen at 400 Hz

Fig. 9 .
Fig. 9. Displacement scatter plot and Bland-Altman plot for XYZ displacement estimation.The scatter plots on the left show the correlation between the actual and estimated values.On the right are the Bland-Altman plots evaluating the agreement between the actual and estimated displacement data.The graph displays a scatter plot of the differences (coordinate) against the averages of the two measurements (abscissa).The black line represents the bias.The dashed lines represent the limits of agreement, defined as ±1.96SD of the differences.The SD and the bias are shown.(a)-(c) Displacement scatter plots for XYZ, respectively.(d)-(f) Displacement Bland-Altman plots for XYZ, respectively.

Fig. 10 .
Fig. 10.Force scatter plot and Bland-Altman plot for XYZ force estimation.For figure specifics, please see Fig. 9, as it has the same information except for the change from displacement to force.(a) X force scatter plot.(b) X force Bland-Altman plot.(c) Y force scatter plot.(d) Y force Bland-Altman plot.(e) Z force scatter plot.(f) Z force Bland-Altman plot.

Fig. 11 .
Fig. 11.Response to vibration in the Z-axis at 400 Hz and a shaker input signal amplitude of 650 mV.The signals are normalized using a unity-based normalization for comparison.(a) LiVec intensity estimate and measured input displacement at compression −0.65 mm.(b) LiVec intensity estimate and measured input displacement at compression −1.6 mm.

Fig. 12 .
Fig. 12. Response to a vibration in the Z-axis at 400 Hz and a input amplitude of 650 mV.(a) and (b) PSDs at compressions 1 and 2, respectively, for the normalized LiVec intensity estimate and the normalized measured displacement input from the laser, with insets showing the dominant peak at 400 Hz.(c) and (d) PSDs for compressions 1 and 2 of the LiVec intensity estimate, showing the dominant peak at 400 Hz, in decibels.

Fig. 13 .
Fig. 13.Vibration transfer function magnitude graph of the LiVec showing the LiVec sensor response to approximately sinusoidal vibrations at frequencies in the range of 50-950 Hz.Various input amplitudes of the vibrations produced as an input to the shaker motor, specifically 50, 500, and 650 mV, were tested; this increase of input signal to the shaker is done to span the frequency range, since, for a given input amplitude, the displacement magnitude achieved by the shaker drops off with increasing frequency due to its inertia.(a) LiVec transfer function magnitude expressed in decibels (dB) at compression: −0.65 mm.(b) PSD LiVec magnitude and the noise floor at compression: −0/65 mm.(c) LiVec transfer function magnitude expressed in dB at compression: −1.6 mm.(d) PSD LiVec magnitude and the noise floor at compression: −0/65 mm.The LiVec sensor shows that with a smaller sinusoidal input displacement, it is only responsive up to 500 Hz, because, after this, the liVec intensity estimate was in the noise floor, meaning a vibration could not be identified.The larger input amplitudes could only be tested at higher frequencies (around 200-Hz upward) due to the testing probe not remaining in contact with the pillar at lower frequencies.

Fig. 14 .
Fig. 14.Light angle sensor raw output channels throughout the temperature increase from 20 • C to 60 • C. (a) Test 1-the sensor in a neutral position.(b) Test 2-the sensor in a compressed position.X1, X2, Y1, and Y2 represent the four photocurrents output by the ADPD2140 light angle sensor.

Fig. 15 .
Fig. 15.(a) Both the resultant XYZ displacement from temperature test 1 and the rebiased XYZ displacement.(b) Both the resultant XYZ force from temperature test 2 and the rebiased XYZ force.
Manuscript received 15 May 2023; revised 29 June 2023; accepted 29 June 2023.Date of publication 14 July 2023; date of current version 14 September 2023.This work was supported by the Science Foundation Ireland (SFI) President of Ireland Future Research Leaders Award under Grant 17/FRL/4832.The associate editor coordinating the review of this article and approving it for publication was Dr. Rui Min.

TABLE II FORCE
ERROR TABLE: THE OVERALL BIAS (MEAN ERROR), PRECISION (SD), AND MAXIMUM (TEST DATASET) 3-D FORCE all at −1 mm Z displacement.The estimated displacement obtained from the multivariate regression model follows the actual displacement very accurately [see dashed versus solid lines in Fig.

TABLE III COMPARISON
OF CHARACTERISTICS OF SIMILAR TACTILE SENSORS, FOCUSING ON SIZE FEATURES.THE SENSOR SIZE REFERS TO THE WHOLE SENSOR INCLUDING CASING.SIMILARLY, THE HEIGHT OF THE SENSOR REFERS TO THE OVERALL HEIGHT OF THE SENSOR INCLUDING CASING.STANDARD DEVIATION IS REFERRED TO AS SD.NOT APPLICABLE IS REFERRED TO AS N/A.INFORMATION NOT AVAILABLE IS DEFINED AS -