Estimating the Carbon Footprint of Small Language Models Powering Edge Device Applications

A new trend in generative AI has risen: small language models (SLMs). These models are built to run on edge devices, including smartphones, tablets, and other devices without access to powerful GPUs or large amounts of computing power. Many recent advancements, including Apple's M4 chip and its implementation in the relatively small iPad Pro, have enabled SLMs to increase in quality and performance as devices can handle larger numbers of parameters. Many analyses of the energy consumption and carbon footprint of training language models have been conducted in the past, but these studies typically neglect the cost of using the models after they have been trained and distributed. While the cost of training these models is significant, it is typically less frequent compared to the usage phase, which is a continuous process. In this paper, we aim to quantify the carbon footprint of language models running solely on CPU power. We measured the energy consumption of seven locally run models of various sizes. The data showed that a model's energy cost is strongly correlated with its size. We then converted the energy consumption data into carbon emissions data using carbon dioxide emissions factors reported by various environmental organizations. Then, using the benchmark scores of several of the models tested, we were able to show a tradeoff between model output quality and carbon emissions. This allows users of these SLMs to apply the data when deciding which model to use in their application, depending on their priorities.