Energy-Efficient Implementation of YOLOv8, Instance Segmentation, and Pose Detection on RISC-V SoC

This article introduces a programmable Artificial Intelligence Visual System on Chip (AIV-SoC), optimized for YOLOv8 end-to-end execution. It operates on input image resolution of $352\times 352$ to perform object detection, instance segmentation, and pose detection, achieving notable Frames per Second (FPS) of 67.1, 55.2, and 64.9, respectively. The AIV-SoC is founded on the RISC-V architecture and meticulously crafted utilizing the LLVM-C2RTL toolkit. Neural network layer scheme optimization, piecewise linear function (PLF) approximation, multi-cycle path, 8-bit fix-point data per-group quantization and 4-pattern transposed convolution are introduced to enhance its versatility and power efficiency. Notably, the implemented model exhibits superior power efficiency in comparison to other GPU and FPGA platforms. This characteristic envisions practical applications across industries. The presented AIV-SoC platform, with its efficiency, opens up possibilities for deployment in resource-constrained environments such as healthcare, agriculture, and surveillance. This potential redefines the landscape of object detection, instance segmentation, and pose detection in real-world scenarios, providing insight into a future where these technologies play a pivotal role in reshaping how we interact with machines and interpret visual information.