Teleport: A High-Performance ShiftNet Hardware Accelerator with Fused Layer Computation

In this paper, we introduce a high-performance ShiftNet-optimized hardware accelerator called Teleport. ShiftNet replaces the standard convolutional layers with zero-flop-based shift convolution and pointwise convolution to reduce the number of computations. However, previous hardware acceleration approaches do not support the shift convolution, and hence they mapped the shift operation to the
$3\times 3$
convolution, and thereby the shift layer still shows the same number of computations as the conventional convolutional layers. To mitigate such a limitation, we first fuse the shift and convolutional layers without modifying the original configuration of the ShiftNets, and the fused computations are accelerated using a custom address translator, a systolic loader, and a systolic array. Our work improved the performance by
$6.1-103\times$
over the previous hardware acceleration approach on the ShiftNet benchmark.