Giant Could Be Tiny: Efficient Inference of Giant Models on Resource-Constrained UAVs

Giant models, characterized by their billions or even trillions of parameters, has demonstrated unprecedented capabilities in handling complex tasks on artificial intelligence (AI)-driven unmanned aerial vehicles (UAVs), such as disaster relief, aerial navigation, and manipulation. However, there is an open challenge about the mismatching between the massive computation and memory requirements of giant models and the limited resources on UAVs. Existing works either pose privacy concerns with offloading methods or compromise model accuracy with various model compression techniques. In this article, we fill the gap by exploiting the Mixture-of-Expert (MoE) model architecture that decouples giant models into multiple tiny experts, so that UAVs can dynamically load a few experts that best match their current input. We consider a general scenario of several edge servers feeding experts to multiple UVAs and formulate a core problem of expert selection and UAV-edge association. Due to the high complexity of this problem, we propose a solution, termed GESolver, based on graph learning, which automatically solves the problem by learning the complicated interaction between edge servers, UAVs, as well as their required experts. We evaluate our proposed method with three popular MoE-based models under various problem settings. The experiments demonstrate that our proposed method can significantly outperform other baselines.