HRCache: Edge-End Collaboration for Mobile Deep Vision Based on H.264 and Approximated Reuse

To accomplish computationally intensive visual tasks on mobile devices with limited memory and computation capability, a common solution is to offload the tasks to the edge with more powerful computing resources. Nevertheless, due to the extra communication, even edge computing is challenged by the growing demands for real-time interactive tasks (e.g., VR and AR), which are generally performed by deep CNNs with heavy computation. In this paper, we design a novel system called HRCache to reduce the end-to-end latency by effectively recompressing video data and reusing cached inference results on the edge. As similar image regions have been indicated in the offloading video coding, the edge can approximately infer these regions with results stored on the edge to save the time for calculating these regions. Moreover, since the data in these regions do not require calculation, the transmitted data can be further simplified to reduce transmission latency. Furthermore, HRCache can quickly and continuously adjust the coding parameters to adapt the accuracy loss and overall latency with application requirements. Compared with the original offloading schemes, HRCache significantly reduces the average latency, about 13.60% to 18.83%, at little accuracy loss of 1.25% in Top-1 accuracy for classification and 0.135 in IoU for object detection.