Federated Closed-Loop Learning for Cross-Modal Generation of Tactile Friction in Tactile Internet

Tactile Internet, as a novel industrial network, allows fully immersive multisensory remote exploration of real or virtual environments. An important technological aspect in tactile Internet is the acquisition, compression, transmission, and display of haptic information. This article focuses on the cross-modal acquisition of fingertip’s tactile friction from visual measurements. In tactile Internet applications, these tactile friction data are transmitted to surface haptic devices for high-fidelity haptic rendering of shapes and textures on touchscreens. To ensure the reliability and latency for such tactile friction acquisition, we develop a federated closed-loop learning (FedCLL) method that is based on the optimized federated learning and the closed-loop learning. The former builds the global model in the centric server, by utilizing deep reinforcement learning to determine aggregation weights of local tactile devices, which improves the acquisition accuracy; The latter generates tactile friction for local devices, by exploring feedback mechanism to achieve improved accuracy and reduced complexity. The proposed FedCLL is numerically evaluated, using HapTex dataset. The results show that FedCLL outperforms existent methods in both acquisition accuracy and computational complexity.

I. Introduction

Tactile Internet, as a novel industrial network, provides real-time control, touch sensing and actuation information through sufficiently reliable, responsive and intelligent connections [1]. This Internet allows fully immersive multisensory remote exploration of real or virtual environments where users can see, hear and, in particular, feel remote objects [2]. The use cases of tactile Internet span over many industrial fields, especially for teleoperation and robotic systems [3]. An important technological aspect in tactile Internet is the acquisition, compression, transmission, and display of tactile data [2]. This article focuses on the acquisition of fingertip’s tactile frictions. In tactile Internet applications, these data are transmitted to surface haptic devices for high-fidelity haptic rendering of shapes and textures on touchscreens [2], [4].