An Improved Lightweight Linear K-value Transformer

In this paper, an improved Transformer network is proposed, which reduces the overall computation of the network by nearly 50°/o while still maintaining good network performance while only retaining K and V values. Experiments show that with Swin Transformer as the backbone network, the improved method proposed in this paper can reduce the training time and testing time while maintaining high accuracy.