AI on the Edge: Architectural Alternatives

Since the advent of mobile computing and IoT, a large amount of data is distributed around the world. Companies are increasingly experimenting with innovative ways of implementing edge/cloud (re)training of AI systems to exploit large quantities of data to optimize their business value. Despite the obvious benefits, companies face challenges as the decision on how to implement edge/cloud (re)training depends on factors such as the task intent, the amount of data needed for (re)training, edge-to-cloud data transfer, the available computing and memory resources. Based on action research in a software-intensive embedded systems company where we study multiple use cases as well as insights from our previous collaborations with industry, we develop a generic framework consisting of five architectural alternatives to deploy AI on the edge utilizing transfer learning. We validate the framework in four additional case companies and present the challenges they face in selecting the optimal architecture. The contribution of the paper is threefold. First, we develop a generic framework consisting of five architectural alternatives ranging from a centralized architecture where cloud (re)training is given priority to a decentralized architecture where edge (re)training is instead given priority. Second, we validate the framework in a qualitative interview study with four additional case companies. As an outcome of validation study, we present two variants to the architectural alternatives identified as part of the framework. Finally, we identify the key challenges that experts face in selecting an ideal architectural alternative.