Efficient Channel Search Algorithm for Convolutional Neural Networks Based on Value Density

Convolutional Neural Networks (CNNs) have exhibited remarkable success in various vision tasks. The allocation of channels within each layer significantly influences CNN performance. Despite the acknowledged importance of channel configurations, achieving an optimal distribution that balances computational efficiency and model accuracy remains challenging. In this paper, we initially transform the CNN training problem into an analogous knapsack optimization problem, incorporating the loss sensitivity criteria. Subsequently, we introduce the concept of value density, employing greedy algorithms, to accurately quantify the improvement in accuracy achievable by increasing the unit FLOPs on every layer. This fosters a proficient exploration and optimization of channel configurations within convolutional layers. Additionally, the efficacy of the proposed methodology has been substantiated through both theoretical derivations and experimental validations by the research team, outperforming popular pruning methods under equal-scale FLOPs conditions. We hope that the integration of value density into channel search algorithms will contribute to the development of more powerful CNNs.