Box Aggregation Distillation: Fractal Feature Aggregation for Model Distillation With IoT Potential

In the context of Internet of Things (IoT) deployments, edge nodes are commonly constrained by limited computational and storage resources, posing resource constraints as a key challenge in edge deployment strategies. Graph knowledge distillation (GKD) effectively extracts and transfers knowledge from large teacher models, enabling model compression and performance enhancement. This provides robust solutions for edge deployments in applications, such as autonomous driving systems, real-time fault detection, and wearable sensors. Nevertheless, current GKD methods have limitations in fully exploiting the multiscale information and deep feature potential of the networks, limiting the improvement of distillation performance. Therefore, we propose a novel graph neural network distillation method based on multiscale fractal feature extraction, termed box aggregation distillation (BAD). BAD divide the network into boxes based on different scales and aggregates features of the nodes within the box to extract the deep structure information. Furthermore, we introduced the Box Aggregation Loss to quantify the differences in intermediate layers between the two models. By minimizing this disparity, we effectively transfer multiscale and deep network knowledge from the teacher model. Experimental results indicate that the BAD method improves the F1 score by 0.51% to 5.83% compared to baseline methods while maintaining model compression. Additionally, it accelerates convergence speed during the warm-up phase by at least 2.96%.