ViART: Vision-Based Soft Tactile Sensing for Autonomous Robotic Vehicles

Navigating and interacting in a narrow space, especially where visual and range sensors do not work, is challenging for autonomous robotic vehicles. In such a scenario, collisions with obstacles and other vehicles are inevitable; thus, effective proximity detection systems are required for safe robot navigation and interactions. This article proposes an innovative design of vision-based soft tactile sensing system for autonomous robotic vehicles (ViART). ViART has a silicone rubber barrel-shaped skin. A set of markers interspaced around the inner surface equator of the skin is monitored by a fish-eye camera installed at one end of the barrel. Displacements of these markers are measured to perceive the physical interaction of tactile sensor and surrounding objects. Contact angle and force of the tactile sensor were estimated and evaluated through a series of experiments, achieving a mean absolute error of 1.12$^{\circ }$ and 0.12 N, respectively. We demonstrated the capability of a developed sensing system with experiments of multiple autonomous mobile robots without visual feedback, showing its capability to navigate in challenging environments, such as narrow spaces, clustered obstacles, and a mix of static and dynamic obstacles. Moreover, with these experiments, we reveal how unique characteristics of the skin including 360$^{\circ }$ multicontact detection and force measurement help leveraging the capacities of obstacle avoidance and navigation of multiagent robotic systems. Tactile-based multicontact sensing capacity provides information on the relative velocity for the robot to avoid obstacles efficiently, whereas tactile-based force estimation enables the robot to avoid obstacles by pushing obstacles away. We expect this contribution may pave a way to implement soft tactile sensing for robust, efficient, safe interaction, and navigation of autonomous vehicles in restricted environments.


I. INTRODUCTION
S ENSE of touch is crucial for creatures to assess and adapt to their environments.Through natural evolution, animals develop unique tactile modalities suitable to their habitats.Generally, the sense of touch is perceived via mechanoreceptors beneath the skin surface, which relay signals conveyed through synaptic pathways to be processed at the brain.Thus, the recognition of applied force and resultant skin deformation and vibration is critical to the sense of touch, so-called tactile perception.The sense of touch includes not only passive perception of incoming stimuli but also active exploration by the body to obtain information about environmental conditions.For instance, humans may press or move their fingers over the surface of an object to assess its tribological properties via human mechanoreceptors, such as roughness, hardness, and wetness [1], [2].For animals, sensing activity is the key of their reaction to surrounding environments.Rodents sweep their whiskers to explore their surroundings and determine their location in dark, narrow spaces [3].Seals use vibrissae (whiskers) to assess and follow water trails left by fish when hunting for prey [4].
In robotics, sense of touch is also useful for recognition among autonomous and smart systems.There have been many attempts to mimic sense of touch found in nature and apply it to robots, especially for the implementation of dexterous grasping and manipulation [5].With the recent emergence of soft robotics, robotic tactile sensing has been extended to other parts of the robot body for the comprehension of interactions between the robot and its environment [6].In addition, researchers have addressed the role of tactile sensing in localization [7] and texture classification [8].Autonomous mobile robots have mainly been developed to operate in complex or hazardous environments [9].To navigate complicated scenarios, autonomous robots are usually equipped with multimodal sensory systems, such as LiDAR, infrared, GPS, or vision systems [10], [11].However, such ranging and visual sensors cannot detect obstacles in close proximity, so mobile robots still encounter collisions with obstacles or peers when operating in swarms [12].In particular, robot swarm is a novel approach to controlling a large number of robots in cooperative tasks, e.g., cooperatively carrying an object.Such visual and ranging sensors are commonly used in mobile robots for swarming operations.However, such sensors cannot detect obstacles at a proximity range, which is needed for robot navigation in narrow areas [13].In addition, these visual and ranging sensors do not provide 360 • sensing capability important for interaction among peers in swarm or cooperation modes.
Motivated by the limitations of existing sensing systems, we introduce the design and implementation of a vision-based soft tactile sensing system for mobile robot navigation and interaction.Equipped with a soft tactile sensor, risks of unexpected collisions can be significantly reduced, so that the robot is capable of navigating in narrow areas as well as physically interacting with its peers.In this article, we present the design, implementation, and analysis of the fabricated soft skin tactile sensing system to demonstrate its feasibility and potential applications through a series of experiments.The contributions of this research are summarized as follows.
1) A design and fabrication process to construct a visionbased soft tactile sensing system with compact, modular, and scalable properties allowing seamless integration into mobile robot platforms (see Fig. 1).Besides, the sensor also provides 360 • sensing force measurement and multicontact detection enabling the robot to react with its operating environment.2) A simplified method for the extraction and estimation of the contact location as well as normal contact force/depth based on inherent morphological properties of the soft skin.Our sensor achieves an overall mean absolute error (MAE) of 1.12 • and 0.12 N for contact angle and force estimation, respectively, at high sampling frequencies (18)(19)(20)(21)(22)(23). 3) Open-source software and hardware of the entire sensing system, thereby enabling further sensor development in robotics.The rest of this article is organized as follows.Section II discusses related work.Section III presents the design, principles, and method of extracting tactile data from captured images.Section IV evaluates tactile sensing of ViART in both static and dynamic cases of physical contact.Section V demonstrates a ViART-based mobile robot navigating a narrow space alone and with peers.The fabrication, calibration, and accuracy of ViART are discussed in Section VI.Finally, Section VII concludes this article.

II. RELATED WORK
Sense of touch is undoubtedly crucial for many robotic systems.Therefore, many have been designed to assess physical properties, such as contact force and contact location through touch between robots and environments.Among touch detection principles, capacitive [14], [15], [16], resistive [17], [18], [19], [20], and optical [21] ones have been widely utilized.For robot applications, tactile sensors are usually combined with soft coverings to enhance the mechanical response, utilizing the topological/morphological specifications of the materials [22].However, such sensors have been applied for small-area devices, such as robotic hands/fingers, whereas the implementation of this method of tactile sensing at a larger scale poses many challenges.In large-scale scenarios, numerous sensing elements with sophisticated electrical wiring systems are necessary, which may result in a burden of data collection, data processing, or even durability and maintenance.
Recently, vision-based tactile sensing has emerged as an efficient method of obtaining tactile information, thanks to computer vision and machine learning approaches [23], [24], [25], [26], [27].Yuan et al. [28] proposed a gel-based sensor named GelSight that uses a transparent gel-covered protrusion to measure the shape and texture.A camera captured the deformation of gel caused by the object's surface.Then, these images were analyzed using computer vision algorithms to create detailed 3-D maps of the object's surface.However, the gel surface could be degraded over time, which can affect the performance of the sensor.Using a similar working principle with GelSight sensor [28], Padmanabha et al. [25] successfully fabricated a fingertip for robotic manipulation with robotic hands, which can detect multidirectional deformations.This design gives the robot a larger sensing area than another vision-based tactile sensor.The OmniTact sensors are more robust but still have limitations in terms of their durability.Typical designs proposed for small scales, such as robot fingers and hands, feature a camera embedded inside soft skin to track movements of markers upon physical contact or deploy machine learning techniques to extract the force information.For large-scale tactile sensors, Duong et al. [29] introduced an artificial human arm design with two fish-eye cameras set up at two ends of the sensor to extract stereo images of skin deformation.The external contact force was synthesized based on the deformation of the skin, which was extracted from two captured images and a finite element method (FEM)-based model of the skin.However, this system required high computational costs, and the sampling rate was as low as 12 Hz for 30 frames per second (fps) cameras.To the best of the author's knowledge, there has been no attempt to integrate such sensing principle into mobile robot systems with 360 • sensing force measurement and 360 • sensing multicontact detection.Thus, we expect that the proposed system paves the way for developing mobile robot research of tactile-sensing systems that allow mobile robots to interact softly with surrounding obstacles and facilitate robot control with tactile-sensing feedback.

A. Design of a Tactile System for an Automobile Robot
This work aimed to develop a tactile sensor module with a simple structure, high accuracy, and scalability enabling reliable integration into various models of autonomous robotic vehicles, especially for small-size robots.To assess tactile information, the sensor should obtain information about contact force, such as magnitude, location, or contact depth.Also, it should be possible to acquire tactile information in any direction from where there may be possible contact between the robot body and its surroundings.To achieve these tasks without the complexity of multiple sensing elements, a vision-based sensing system is used.
Fig. 1 shows the general design of the tactile sensor integrated with a Zumo base robot, which is a typical commercial small-size mobile robot platform.Other platforms can be used similarly.A black barrel-shaped soft skin (silicon-rubber Dragon Skin 30, Smooth-On, USA) is enclosed at two ends connected by a transparent acrylic tube within the body, which maintains the soft skin in a barrel shape.On the inner surface of the soft skin, a set of 24 red markers is distributed around the skin equator, which is monitored by a web camera (ELP-USBFHD06H-L-180 USB Camera, 30 fps).The camera is opposite the Zumo base and observes the markers through the transparent tube.A pair of flexible wires is loosely attached along the soft skin to establish communication between a minicomputer (on the upper base) for image acquisition and an Arduino (on the Zumo base) for movement control.The wire is sufficiently soft and small to avoid influencing the mechanical properties of the soft skin.
A barrel-shaped skin was considered best to observe all possible physical contact with the surroundings and to enhance the protection of the Zumo base.Also, since contacts were assumed to occur radially at the equatorial area, it was deemed unnecessary to extend the markers across the entire inner surface of the skin.Thus, markers were equally interspaced along the equatorial area, as shown in Fig. 1.Depending on application requirements, more markers may be included to assess contact at other locations.

B. Fabrication
In this section, we introduce the process for the fabrication of the barrel soft skin with markers.The method can be applied to create similar shapes of various sizes.The soft skin is made by a molding process, which is shown in Fig. 2, and explained as follows.
1) Molds designed by 3-D software (Solidworks, Dassault Systémes) were fabricated by a 3-D printer (M300, Zortrax).2) A red silicone rubber mixture (Dragon Skin 00-20 and red pigment, Smooth-On) was prepared and filled into holes in the inner mold to shape the markers.3) A silicone compound was mixed from 140 g of Dragon skin part A, 140 g of Dragon skin part B, and 2 g of black pigment.4) While stirring the silicone mixture, bubble formation could disrupt the uniformity of the soft skin.To prevent this problem, it was degassed for 5 min using a vacuum dryer (AVO-200NB-CR, AS ONE) to eliminate bubbles.5) After that, the mixture was poured into a prepared mold and left at room temperature for six hours until entirely condensed.6) The soft skin was separated from the mold and assembled with other parts to produce the tactile sensor module.

C. Vision System
Fig. 3 represents the workflow for producing the tactile sensor.In this study, a fish-eye camera based on the pinhole model was utilized to capture images of the markers.Then, the markers are detected in the image coordinate system (OXY ).Their positions are transformed into the skin coordinate system (CXY ), which is used for the tactile data extraction process.The 2-D movement of the markers is constructed as skin coordinate system (see Fig. 4).We utilized image processing tools provided by OpenCV [30] libraries to detect the markers' positions in each image frame, which resulted in the construction of 2-D deformation of the equatorial area.The procedure of image processing contains three main parts as follows.
1) Initially, markers are captured by the fish-eye camera and numbered from i = 0, 1, 2. .., 22, 23.Their original Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.positions M i (x i , y i ) w.r.t skin coordinate system (CXY ) will be recorded.2) When an impact occurs, contact location on the tactile sensor is determined by comparing the markers' original positions M i (x i , y i ) with their current positions M i (x i , y i ).
3) Finally, relying on displacement observed at contact spot, contact force magnitude is estimated accordingly.For ease of computation, the following assumptions were made in this study: 1) Contact with the environment (with a peer or obstacle) occurs only at the equatorial area.2) Contacts are normal w.r.t. the equatorial surface, i.e., contact forces are radial.1) Marker Detection and Labeling: Since the system compares the positions of markers, marker detection and labeling processes are necessary.At first, real-time images (640 × 480 pixels frame size) are acquired to detect and track marker movements.Captured images in RGB space are converted to HSV space for higher marker detection efficiency.RGB color space includes the three color components red, green, and blue; whereas HSV color space includes hue, saturation, value, making HSV more robust to change in external lighting [31].Specifically, Hue values vary less than RGB values in cases of minor changes in external lighting such as pale shadows, etc. Color filters are applied to the HSV images to minimize noise and eliminate unwanted colored areas by turning them into black areas, except markers (i.e., white color areas).Then, the center of each white area is calculated to represent the center of each red marker.To simplify the marker labeling process, marker positions are transformed from the image coordinate system (OXY ) to the skin coordinate system (CXY ).The new center position is calculated by averaging the value of the total of 24 markers' positions in each Xand Y -axis as shown below where with i = 0, 1, 2,. .., 23, N = 24, whereas, M i (X i , Y i ) and M i (x i , y i ) are positions of marker in OXY and CXY , respectively.
Next, the marker positions are converted to polar coordinates (r, θ) using the following conversions: with i = 0, 1, 2,. .., 23, whereas θ and r are angle value and radius, respectively.Finally, the ID of each marker is numbered from 0 to 23, following the counterclockwise direction [see Fig. 4(a)].
2) Contact Location Estimation: For the mobile robots equipped with a tactile sensor to operate in an unstructured and narrow space where robots frequently interact with surrounding objects, having knowledge of contact location is essential.For our sensor, the region of soft skin deformation is detected and the contact location is determined by analyzing the displacement of the markers.Note that, the location of a contact may or may not coincide with the location of a marker.Thus, it is necessary to specify the marker that is closest to the contact point.First, windows of three adjacent markers (shaded ones in Fig. 4) are examined to find the group that contains the most significant movement of the middle marker.Specifically, in one calculation cycle of each captured image of markers, a window containing a group of three adjacent markers begins at the first group of three markers {0, 1, 2} and continues shifting one unit (marker) until reaching the last group {23, 0, 1}.Once the displacement of the middle marker of a specific group is larger than the others, the algorithm will assign a potential contact event to such a group.This method allows for the simultaneous detection of multiple contact areas.
After detecting the contact group(s), the contact location will be determined by analyzing the relative position among markers in group(s).First, it is assumed that the contact location is within the region between the middle marker (M 2 ) and the marker with the second-highest displacement distance (M 1 ), as shown in Fig. 4.Then, two lines (l 1 = M 1 M 1 and l 2 = M 2 M 2 ) connecting original positions (M 1 , M 2 ) and current positions (M 1 , M 2 ) will be created.Mathematical expressions for these lines can be seen as follows: Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.with M 1 (x 1 , y 1 ) and M 2 (x 2 , y 2 ) being the original marker positions, whereas M 1 (x 1 , y 1 ) and M 2 (x 2 , y 2 ) being the current marker positions.
The intersection point of these two lines is defined as an external stimulation source-notated as G (see Fig. 4).Based on the assumption of radial contact in Section III-C, the contact point E is located at the intersection point between the GC line and the skin great circle.Due to the characteristics of the polar coordinate system, any point on the line GC has the same angle value (θ) except the origin C. Therefore, the result of contact location is defined by the θ of point G w.r.t. the origin.Furthermore, in this study, to simplify the visualization of the contact location, the contact point E is hypothetically on the equator of the skin.
3) Contact Force Estimation: As well as the contact location, the contact force plays an important role in physical interactions.Several methods have been proposed to estimate it, including FEM and machine learning techniques.The FEM approach uses the mathematical model of soft skin to estimate the applied force, whereas machine learning techniques rely on the amount of data for training the model.However, both methods contain several drawbacks.The FEM method demands an intensive computational cost.Meanwhile, the machine learning technique requires an expensive cost in data collection, especially in multiple contact detection, which requires a complex data collection system.To address these challenges, this article proposed a method to shorten the deployment time of the contact force estimation algorithm for the proposed tactile sensory system.The main principle of this solution is to map the displacement of the middle marker (in pixels) and contact angle (in degrees) to the magnitude of contact force (in newtons).As a result, in this study, the sensing rate is around 18-23 Hz for a 30 fps camera.
First, visual cues obtained from Section III-C2 (contact location and displacement of markers) and ground-truth contact force (measured by IMADA ZTS-5N force gauge) were collected by conducting a systematic data collection experiment.In detail, a set of calibration processes were designed, which are shown in Fig. 5.The experimental apparatus comprises a linear stage motor (PG750-L05AGUA, Suruga Seiki, Japan), a servo motor (Dynamixel XM540-W270-R), a force sensor (IMADA ZTS-5N force gauge), and a flat indentor (with surface  area of 100 × 100 mm 2 ).The tactile sensor is mounted on the servo motor for rotation generation, i.e., different contact angles from the indentor.The x-linear stage was controlled to bring in translational movement of the indentor toward the soft skin, while the indentor was attached to the force gauge to measure contact force.During experimental trials, a desktop computer was employed to acquire tactile data and contact force values from the force gauge.Finally, a fitting function (a third-order equation in Fig. 3, i.e., F = f F (Δ N , θ)) representing the correlation between visual cues and ground-truth data will be formulated using MATLAB (see Fig. 6).It also means that, if the moving distance of the marker that is nearest to the contact Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.point Δ N and contact angle θ are given using the algorithm in Section III-C2, the contact force F is estimated using the function F = f F (Δ N , θ).

IV. EVALUATION OF SENSOR PERFORMANCE
In this section, we describe how the sensing performance of ViART was evaluated in both static and dynamic cases of physical contact.Then, we showcased how the completed ViART was utilized to guide a group of mobile robots traveling through a narrow space.

A. Static Case
In this evaluation, the indentor made contact at two predefined locations, one is at a marker's location and the other is between two adjacent markers.The experiment setup is similar to the one used in Fig. 5.The indentor was driven toward the soft skin at a desired contact location to achieve a certain contact depth.The contact depth was within a range of 0-10 mm with a step of 1 mm.This process was conducted repeatedly five times for each contact position.Then, a servo motor rotated the tactile sensor concentrically in steps of 1 • to change the relative location between the indentor and the soft skin.The process was accomplished until the rotation angle of the servo motor reached 360 • .During the experiment, the reference data including contact angle θ (equivalent to the servo motor rotation angle) and contact force measured by the force gauge were collected simultaneously.These results will then be compared with the estimated values acquired from the proposed method in Section III-C.The results of the evaluation process were split into five groups based on the contact depth.The details are given below.
1) Contact Location: Fig. 7(a) shows the deviation of the actual and estimated contact angles θ, which reveals the location of contact in the equator circle.According to Fig. 7(a-1), the precision of our sensor was relatively high (error around ±1.5 • ) when the contact depth at a specific marker went over 2 mm.Meanwhile, the early stage of the indention (i.e., 1-2 mm) witnessed an error up to ±2 • for all groups.This phenomenon could be related to the creation of the point G as mentioned in Section III-C.More specifically, the stability of the point G is linearly proportional to the contact depth (or the distance between M 1,2 and M 1,2 ).On the other hand, a similar performance was also observed when the contact spot was between two adjacent markers [see Fig. 7(a-2)].The overall MAE of contact angle estimation is 1.12 • with a standard deviation of 0.6 • .
In general, the reason for such an error could lie in the following points.First, in this research, tactile images are at low resolution (640 × 480 pixels) to reduce the computational cost.A higher image resolution can be expected to increase the accuracy of contact force and location estimation.Second, the thickness of the fabricated soft skin may be inconsistent over the entire surface due to the fabrication process.Therefore, deviation varies over different contact angles, even though overall errors are within an acceptable range.Mass production using injection molding is expected to overcome this limitation, thus increasing sensing accuracy.
2) Contact Force: The evaluation results for contact force estimation are shown in Fig. 7(b), where the maximum error is around 20%.The deviations (errors) between the estimated contact force and the ground-truth value at various contact depths were plotted together over several selected locations on the equator.In detail, the evaluated locations were selected at the Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.location of the marker [see Fig. 7(b-1)], and the location between two adjacent markers [see Fig. 7(b-2)].The overall deviations were similar for both the cases.Similar to the contact location estimation, the force deviation increases when the contact depth is from 1 to 2 mm, whereas it tends to be smaller at larger contact depths.This is understandable since the error of contact location estimation for this small-depth group [shown in Fig. 7(a)] was also the largest among groups.The overall MAE is 0.12 N with a standard deviation of 0.045 N.

B. Multicontact Detection
The proposed tactile sensor is integrated to multimobile robot systems aiming to operate in narrow spaces, such as complex structures.Therefore, the tactile sensor has a high possibility of perceiving multiple external contacts simultaneously in different contact locations.In this scenario, the ability to detect multicontact points simultaneously and sensor response time in such cases are critical factors for the robot adaptation.To assess these features of the proposed tactile sensor, we simultaneously forced four indentors into four arbitrary contacts around the tactile skin.The demonstration of this test is shown in Fig. 8.It is clearly seen that the tactile sensor was able to detect multiple contacts as expected.We experimented with investigating the mutual interactions between multiple contact points.Specifically, we randomly selected a set point with the highest contact force, as measured by a tactile sensor (3.5 N), and performed two touch scenarios: one-sided and two-sided.It is important to note that the contact in two scenarios contacted with the highest contact force (3.5 N).During these experiments, we slowly drafted the contact point toward the set point location, and then measured changes in contact force and contact angle estimation.Any set point's parameter changes indicated the minimum distance between two points at which mutual interactions no longer occurred.As a result, the minimum contact degree must be above 49.54 • between two adjacent contact locations to avoid any effect on the contact force estimation result.

C. Sampling Rate
Another aspect that attracted our attention is the sampling rate of the sensor during normal operation.This was calculated by measuring the time duration of a program cycle.In detail, we determined the time required for the onboard computer to complete tactile data extraction, starting from the time an image was captured until the contact location and magnitude of contact force were estimated.The measurement was conducted with several test conditions including single-contact detection and multiple-contact detection.As a result, the tactile sensor achieves sampling rates from 18 to 23 Hz.In this study, the integrated camera works at 30 fps, so the proposed tactile sensor is limited in achieving a higher sampling rate.Not to mention the fact that the microprocessor used in our system is limited in terms of computational power.Thus, one may use cameras of higher resolution and speed to increase the sampling rate of ViART.

D. Dynamic Case
However, in practical scenarios, the properties of the surrounding are dynamically changed.Therefore, it is worth evaluating the reliability of our sensor system in such challenging circumstances.To achieve that, we tracked the sensor feedback in real time in response to the external pressing at different constant speeds.The force gauge used in the static experiment was replaced with a force sensor (USL06-H5, Tecgihan, Japan) that can provide a higher sampling rate (200 Hz).The indentor was controlled to contact the soft skin at speeds ranging from 10 to 45 mm/s.Fig. 9 reveals that the estimated values were close to those of the ground-truth values (from the force sensor).It can be seen that when the indentor speed was up to 45 mm/s (black lines), the difference between the estimated value and the ground-truth value increased.Therefore, with the current camera and onboard computer specifications, the tactile sensor can achieve stable performance with contact speeds below 40 mm/s.

V. SHOWCASES
This section aims to highlight the unique characteristics of 360 • sensing tactile sensors by integrating them into multiple autonomous vehicles in a complex navigation task.

A. Overview
A ViART-based differential-driven mobile robot system is expected to operate in a complex and narrow environment Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.Fig. 10.Three states of the robot in different touch conditions.In state 1, the robot moves to the goal.Once the collisions are detected, the robot behaviors will be controlled by states 2 and 3.

TABLE I SWITCHING CONDITIONS FOR DIFFERENT NAVIGATION MODES ARE BASED ON THE MAXIMUM SAFE OPERATING FORCE OF THE ROBOT (F MAX ), THE MEASURED VELOCITY OF THE ROBOT ( − → V R ), TIMER VALUE IN SECONDS (T ), AND A SHORT TIME WINDOW (t t ) FOR THE ROBOT TO DETERMINE WHETHER OBSTACLES CAN BE MOVED SAFELY
randomly filled with movable and unmovable (fixed) obstacles, which are unknown to robots.The objective of robots is to overcome multiple collisions among the robots and obstacles, and navigate toward the desired destination robustly and efficiently.Each robot knows the approximate direction toward the goal.Technically, the OptiTrack system is used to acquire robot's relative location and orientation w.r.t.its goal so that V G can be identified.Furthermore, robot location and orientation are also used to correct any deviations in rotation or movement.Note that the optical system does not provide any information regarding the contact/collision of the robots.During the robot's movement, the tactile sensor continuously monitors for potential collisions with either obstacles or robots.If collisions are detected, the robot behaviors described in Fig. 10 and Section V-B are triggered to react appropriately.

B. Touch-Based Obstacle Avoidance
In this article, we present an obstacle avoidance algorithm that utilizes tactile sensor feedback and the state machine approach, as shown in Fig. 10.The behavior of each robot during the task is defined by one of the following three states: move to goal (state I), move backward (state II), and move along objects (state III).In the initial state (state I), a robot navigates to the goal using a simple PI controller that steers the robot axis.When the robot's tactile sensors detect at least one collision with an obstacle, the robot's timer is triggered to check the switching conditions described in Table I, and determine whether the robot should continue moving or switch to state II to avoid the obstacles.When there is no collision, the timer is reset.
Each robot is equipped with touch-based obstruction (TBO) conditions determined by the robot's velocity and contact forces.The TBO conditions are based on two criteria: 1) if any contact force exceeds the threshold F max , the robot must immediately avoid the obstacle to prevent damage to itself or to the obstacle, and 2) if the robot cannot move within predefined t t seconds, the robot should avoid the obstacle to reach the goal.The TBO conditions (TBO.1 and TBO.2) enable the robots to navigate the complex environment by pushing movable objects safely without avoiding all objects.By doing that, the robots can move through the environment, while potentially clearing the path for another robot.
Due to the nonholonomic characteristics of a differential-drive robot, it is necessary to split the obstacle avoidance behavior into two states.First, the robot moves backward by the backward vector − → V B for a duration of t r seconds to relax the contact with obstacles (state II).The robot steers its axis to a perpendicular direction and moves forward a short distance to avoid the obstacle.The backward vector − → V B is calculated by summing all contact force vectors defined in Section IV-A2, where − → V B = − → F i .The move along vector − → V B is defined by rotating − → V B by 90 • .These contact force vectors represent the movement of the obstacles relative to the robot frame.For a single obstacle collision, the fastest way to avoid it is to move perpendicular to its relative velocity.In scenarios involving multiple collisions, obstacles in motion toward the robot exhibit higher relative speeds in the robot frame compared with the static objects.Conversely, obstacles moving away from the robot have the lowest relative speed compared with that of the robot frame.Therefore, in this article, we prioritize avoiding the higher relative speed in the robot frame based on the sum of the contact force vector-the higher magnitude of the contact force vector has more impact on the sum than the shorter one.To control the robot's movement, we define the desired velocity of robot j as − − → V Dj based on each state of the robot and the equivalent weight of each component vector α, β, γ using the following equation:

C. Setup
The experimental apparatus includes a central computer (Core(TM) i7-7700 K CPU, 4.2 Ghz (8 CPUs), 16 GB RAM) and an optical tracking system (shortened as OptiTrack), which consists of six Flex-13 cameras as shown in Fig. 11(a).The OptiTrack system provides real-time updates of the robot's position, which are utilized to calculate the robot's velocity and desired direction toward the predefined goal for each robot via the TCP/IP protocol.These feedback signals are used as an alternative to the local inertia measurement of the robot.Each robot in the experiment is equipped with a Raspberry Pi 4B, which receives the feedback signals from the OptiTrack system and the sensor to calculate the pulsewidth modulation (PWM) for the left and right wheels of the Zumo mobile platform.The necessary experimental parameters are presented in Table II.
Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.

D. Multiple Robot Navigation
We evaluated the interaction and navigation of multiple ViART robots tasked to navigate through a complex environment to a goal [see Fig. 11(a)].A typical setup resulting from the paths of four ViART robots is shown in Fig. 11(b).According to this figure, robot 2 had less collision and reached the destination first in 130 s.Once robot 3 escaped the narrow space and reached the destination in 156 s, robots 4 and 1 had more space and quickly escaped the narrow space at 169 and 182 s.During navigation, one robot could potentially push the movable obstacle and ease the navigation for itself and other robots.As shown in Fig. 12(a), robot 1 approached box 2 first and conveniently pushed the box to make room for robot 2 and other robots to navigate through the narrow space.Meanwhile, in Fig. 12(b), robots 2, 4, and 1 navigated an interesting path when they collided simultaneously with walls, obstacles, and other robots, obstructing another robot's movement in the narrow area.In this case, multiple contacts occur simultaneously on the skin of each robot.The backward direction would be leaned more on the higher magnitude of the contact force to avoid obstacles.Analysis of the contact forces in robot 2 reveals that due to its relative speed with robot 4, the contact forces between them are comparatively low compared with the forces resulting from collisions with robot 1 and the box.Consequently, robots 2 and 4 exhibit backward movement, as demonstrated in Fig. 12(b), to alleviate congestion and successfully navigate through the cluttered environment in a narrow space.

A. Fabrication of Soft Skin
The performance of the ViART system is strongly dependent on the fabrication of the soft skin.Current molding methods (3D-printed molds and silicone rubber) enable rudimentary fabrication of the skin.However, due to alignment errors among mold parts, the skin thickness may not be consistent over the entire sensing region (e.g., the equatorial area), causing variation in sensitivity.In the future, an injection molding process could be employed to improve the precision and consistency of the soft skin for mass production of the ViART sensing system.

B. Calibration
The ViART system requires calibration before operation.The proposed calibration method enables the sensor to extract highly accurate tactile information using a low-cost computer and an inexpensive camera.However, inconsistencies among units of the fish-eye lens employed for each ViART sensor require a calibration process to eliminate such differences.And the calibration result of one tactile sensor could not be used for the other tactile sensors.In this article, we reported a semiautonomous calibration setup.We aim to develop a fully autonomous system in the future, especially to apply ViART in a swarm robotic system and toward mass production.Technically, a transformation matrix could be applied to transform the image obtained from a new ViART tactile sensor to match the image of a precalibrated reference one.More elaboration will be conducted in the near future.

C. Accuracy
Several limitations affect the accuracy of extracted tactile data in the proposed system.The type of contact force was assumed to be radial.However, in real work environments, friction occurs when mobile robots interact with objects.Thus, the resultant estimated contact force might slightly differ from the actual magnitude of an external contact force.In addition, when soft skin deforms marginally, the low-profile camera employed in this study could only capture minimal deformation of the tactile skin, leading to inaccuracy and a wide range of estimated contact force fluctuation, especially for shallow contact depths.Another limitation is that the tactile skin was designed with one row of markers (around the equator) dedicated to collisions around this deformable region.If contact occurs beyond this region, collision detection capacity is reduced significantly.Such an external contact would require a higher magnitude of force to displace the markers; therefore, the accuracy of the estimated data would markedly decrease.To improve sensing capacity, one could distribute markers over a larger surface area of the soft skin (see one of our solutions here [32]), and employ a high-specification camera.In addition, deploying an algorithm to detect friction is essential to improve the accuracy of an estimated result.Nonetheless, it could be a burden for low-power minicomputers and reduce the sampling rate of the tactile sensor.

D. Soft Skin Characteristic Highlighted by Demonstration
The proposed tactile sensor enables robots to measure force and detect multiple contacts in a 360 • field, making their performance more efficient, robust, and safe in complex navigation tasks.Avoiding obstacles, including movable ones, can lead to longer distances due to detours.In contrast, pushing obstacles could clear the path for the robot and its peers, increasing the overall efficiency.Moreover, avoiding obstacles can trap the robot in more complex situations, whereas pushing obstacles can help it navigate through clustered environments.For instance, in Fig. 12(a), box 2 obstructs the narrow path for the robots, and pushing it is the only way for the robot system to pass through.This approach can enhance the system's robustness in case some movable obstacles already block the way.However, it is essential to note that the proposed algorithm does not manipulate the box's movement, and it may inadvertently block another robot's path, making navigation difficult.With 360 • multicontact detection, the robot can accurately estimate the location of contacts and avoid obstacles in a decent way [see Fig. 12(b)].The 360 • force measurement enables safe interactions with surrounding objects.It is essential to note that this demonstration also reveals the limitations of the robot's nonholonomic motion constraint, since one can expect that the omnidirectional movement would provide a better response strategy.In this article, we did not aim for an optimal design robot movement and collision-response algorithm, since that is a matter for future work.

VII. CONCLUSION
In conclusion, our article introduces ViART, a vision-based tactile sensor capable of 360 • sensing for autonomous robots on a large scale.Our results show that ViART achieves an MAE of 1.12 • and 0.12 N for contact angle and force estimation, respectively, and showcases the naïve robot behaviors.Compared with machine learning techniques, our method offers an alternative way to detect and extract multicontact information effortlessly and successfully deploy into a low-cost computer with the sampling rate fluctuating from 18 to 23 Hz.Thanks to its 360 • sensing capability, autonomous robots can improve the sensing of their environment, minimize the complexity of robot systems, and facilitate the development of advanced robot control algorithms.

Fig. 1 .
Fig. 1.Design of the integrated tactile sensor autonomous vehicle, which consists of the proposed tactile sensory system (soft skin and minicomputer) and mobile robot base (Zumo base).

Fig. 2 .
Fig.2.Fabrication of the soft skin based on a molding process begins with the design and mold fabrication.Then, a silicone rubber compound is prepared and the markers are formed.Next, the mixture is poured into the mold and left for six hours until it cures completely.

Fig. 3 .
Fig.3.Workflow of vision-based tactile sensor consists of two steps: 1) Marker tracking is where current markers position M i (x i , y i ) is measured, and 2) based on that, contact properties will be estimated in the next step.

Fig. 4 .
Fig. 4. Principle of contact location detection.(a) Analyzing marker movement upon a collision.(b) Markers are revealed after applying color filters.The marker area (red color) shows as a white area.Meanwhile, unwanted color shows as a black area.(c) Contact point E is determined on the equator once the tactile data extraction process is completed.The red markers are equally interspaced around the skin's inner surface equator.(d) Examination of three adjacent markers in one calculation cycle.The shaded block is the window that contains the markers in the detection contact region.This window shifts one unit block until reaching the group of markers that has the ID {23, 0, 1}.(a) Principle of contact location detection.(b) The market positions when contact occured.(c) The illustration of detected contact location in real-time.(d) Examination of three adjacent markers in one calculation cycle.

Fig. 5 .
Fig. 5. Design of the experimental setup for the data collection process and statics collision evaluation, in which an indentor (with surface area of 100 × 100 mm 2 ) mounted onto a linear stage moved toward the soft skin.During the experiment, a desktop computer was employed to acquire the tactile data and contact force values of the force sensor.(a) Experiment setup.(b) Experiment diagram.

Fig. 6 .
Fig. 6.Correlation between external contact forces F and tactile data including contact location θ and moving distance of the marker that is nearest to the contact point Δ N .The data collection experiment was designed to acquire sensory data on the equator of the tactile sensor.

Fig. 7 .
Fig. 7. Comparison between the ground truth and the estimated value.(a) Difference in the estimated contact location at (a-1) the marker and (a-2) the middle point.(b) Difference in the magnitude of the contact force at (b-1) the marker and (b-2) the middle point.(a) Contact location evaluation.(b) Contact force evaluation.

Fig. 8 .
Fig. 8. Multicontact detection with the proposed tactile data extraction method.(a) Markers (white area) are revealed under several color filters.(b) Detected contacts are reflected by four external contact forces F 1 , F 2 , F 3 , and F 4 .(a) The marker positions when contact occured.(b) The illustration of detected multi contact location in real-time.

Fig. 9 .
Fig. 9. Comparison of force measurement between the force sensor and the estimated force of the tactile sensor at different contact speeds.The proposed tactile sensor was tested at speeds ranging from 10 to 45 mm/s.

Fig. 11 .
Fig. 11.(a) Test field for a multirobot system.(b) Whole path of each robot during the experiment.The group of four mobile robots reached the destination based on the feedback from the proposed tactile sensor.(a) Environment experiment.(b) Robot movement path.

Fig. 12 .
Fig. 12. Hardware experiment to highlight unique skin characteristics in a complex navigation task.(a) Robot 1 conveniently pushes the box to its goal, making room for robot 2. (b) Robots 2 and 4 move backward by vector − → V B when the tactile sensor detects multiple contacts.

TABLE II VALUE
OF PARAMETERS SELECTED FOR EXPERIMENTS