DLASE: A light-weight framework supporting Deep Learning for Edge Devices

With the rapid growth of the Internet of Things (IoT), there is a massive number of constrained devices connected to the internet, resulting in the generation of large data volume. The deep learning (DL) has become a promised solution to extract more valuable knowledge from collected data but struggle to execute DL algorithms due to limited computing resources of IoT devices. Leveraging the computation power of cloud computing, we could offload the data to cloud servers for processing and response results to the devices. But, this paradigm leads to a significant increase in latency and security risks. Therefore, there have been many efforts to perform DL algorithms on edge devices, which are close to the data sources. Deploying various DL systems to such edge devices a complex process due to the diversity of DL models and running environment configuration. In this paper, we propose a light-weight framework supporting to deploy deep learning as a service running on edge devices, namely DLASE. There are two key advantages of our proposal. The first one is that we enable remote deployment of various deep learning models to edge devices. The other one is that these models are run as a service to serve requests from IoT devices. To demonstrate the light-weight and flexibility of DLASE, we practically evaluate the framework performances (consumed memory, CPU, latency) by running various combinations of DL models and edge devices.