Repnas: Searching for Efficient Re-Parameterizing Blocks

In the past years, significant improvements in the field of neural architecture search(NAS) have been made. However, in order to improve the performance of the model, many search spaces contain multi-branch architectures, which leads to the gap between the searched constraint and real inference time. In this work, we propose a re-parameterization (Rep) search space based on structural Rep techniques. In the Rep search space, the subnets are multi-branch in training and single-path in inference. Furthermore, RepNAS, a one-stage NAS approach, is present to efficiently search the optimal diverse branch block (ODBB) for each layer under the branch number constraint. Our experimental results show the searched ODBB can easily surpass the manual diverse branch block (DBB) with efficient training.