Assessment of Carbon-Aware Flexibility Measures From Data Centres Using Machine Learning

Potential solutions for unlocking operational flexibility from data centres are investigated. Two types of data centres, in terms of their cooling facilities, are modelled in order to provide an in-depth assessment of potential variations in data centre performance based on their energy consumption and server temperatures. Using synthetic datasets generated from year-long building dynamic simulations under distinct system conditions, four machine learning (ML) models are designed and trained to predict data centre energy consumption and server temperatures. Three different ML algorithms are considered, including random forest, XGBOOST, and multiple linear regression, which all achieve high accuracy ranging from 93.8% to 98.1% for the mean absolute percentage error. The resulting ML models are then utilised to represent a system-wide fleet of data centres, which are integrated within a power system unit scheduling framework for potential demand shifting based on (system) carbon intensity. Five alternative flexibility scenarios are introduced and compared, with the objective of determining the achievable flexibility from a system-wide portfolio of data centres, focusing on concerns relating to server temperatures. On average, a 6.5% load reduction is seen to be achievable for a winter day during periods of high CO2 intensity.


A. Background
Recently, the world gathered in Glasgow to reach a new agreement on tackling climate change, where the majority of This work was supported in part by IES Ltd., in part by Science Foundation participating countries agreed to reinforce their commitments, and to intensify activities in the power and energy sector to reduce related carbon emissions. Ireland, which is often seen as a leader in decarbonising its electricity network, introduced a new target of up to 80% of renewables by 2030, as an important measure to tackle climate change, and to achieve climate neutrality no later than 2050, similar to other EU countries [1]. Meanwhile, the world is witnessing increasing demand for computing resources and data centre power due to digitisation, e.g. IT workloads have rocketed sixfold between 2010 and 2018 worldwide, with ensuing environmental consequences [2]. Electricity demand growth in Ireland, for example, over the next ten years is largely expected to come from new data centres, with projections that they will consume approaching 30% of demand by 2030. In 2020, 11% of total Irish electricity demand was dedicated to the data centre industry, contributing 1.85% of Ireland's total carbon emissions in the same year [3]. This rapid digital transformation of businesses is an obstacle to reaching a zero-carbon EU by 2050, unless initiatives are implemented to make data centres carbon-neutral, or more environmental friendly. For example, the "Climate Neutral Data Centre Pact" was agreed by major industry players and the trade association for cloud infrastructure services and data centres in Europe to achieve climate neutrality by 2030 [4]. Recently, the concept of carbon intensity has garnered significant interest, especially regarding how it should be accurately quantified and how to promote its acceptance by end users to change their consumption patterns. Commonly used by TSOs, the total CO 2 emission produced in the system divided by the total system demand is referred to as the "average emission rate" [5]. So, end uses, and particularly large scale ones, such as data centres, could potentially reduce their consumption during periods of high CO 2 intensity, and shift their demand profile towards periods of lower carbon intensity. Given the relative size of data centres, and their overall share of electrical demand, it is not unreasonable for them to provide a greater than proportionate share of "carbon-aware load shifting". Indeed, obtaining equivalent flexibility from a similarly sized block of demand composed of a varied group of small enterprises, public facilities, and residential customers would be more challenging [6].

B. Literature Review
In recent years, considerable effort has focussed on reducing data centre energy costs, improving energy efficiency and/or This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ reducing carbon emissions, but mostly from an individual data centre perspective [7], [8], [9], [10]. Many challenges still remain in unlocking data centre flexibility, with the most basic approaches relying upon on-site backup generation, uninterruptible power supply units (UPS), or battery energy storage systems. However, utilising UPS systems to provide demand response is typically not possible, unless additional functionalities are incorporated within the power electronics converters [11]. Alternatively, flexibility can be obtained from delay-tolerant IT workloads [12], such that, for example, model predictive control can be applied to optimise data centre operation and increase the system-wide renewables share. A real-time energy management system has also been proposed [9], with the objective of minimising data centre energy costs, but the underlying model only considers limited energy performance details and external power system scheduling influences are not considered [13]. Data centre optimisation within a microgrid has also been investigated, while the potential of using data centres to provide a fast frequency response using UPS and flexible IT workloads has also been evaluated [11].
Similarly, the impact of data centre growth on power system operation has been investigated [10], demonstrating reductions in both electricity price variability and carbon emissions. Microsoft recently introduced a low-carbon scheduling policy for its Kubernetes service [14], while Google has introduced carbonaware scheduling for their data centres by adjusting virtual server capacity based on system-wide carbon intensity [8]. Although potential solutions involving IT workload re-scheduling have been discussed by big tech companies [8], [14], questions still surround the impact of IT workload re-scheduling on other facilities, such as cooling, and the overall impact on CO 2 emissions, power system operation, and even energy markets.
Analysis of data centres either tends to consider them in great detail [15], or relies upon greatly simplified models [10], [13], which essentially treat all types of data centres as being the same. In practice, however, individual data centres are quite varied, especially in relation to their cooling facilities, such that for relatively mild and humid climates, external airflow is typically used to cool the servers. However, if 24/7 cooling is instead required, then the nature of the cooling facilities will change. Servers represent the major data centre load, with cooling facilities coming next, but they are correlated with server temperatures and IT workload. Ultimately, server temperatures are the main concern for safe data centre operation and, hence, they define the main challenge for demand-side management approaches, due to sensitivity towards secure functioning to deliver requested IT workloads.

C. Research Gap & Main Contributions
1) Dynamic Modelling Approaches of Data Centres: A wide range of numerical models have been proposed for analysing building energy performance [16], [17]. For example, computational fluid dynamics (CFD) simulates air flow and temperature distribution in buildings, such as data centres, to study thermal behaviour [18]. However, computational times are long, and real-time energy management is not feasible [16]. Alternatively, various numerical approaches have evolved, including a Laplacian model to predict temperature distributions [19]. Building and energy simulation models, such as IES-VE, Energy Plus [15], DOE-2, have also been used to design and optimise buildings and HVAC systems, and to determine baselines and probable retrofit energy savings. They can predict a building's thermal performance, but the slow processing time is again a barrier for modelling a fleet of buildings. Resistance-capacitance (RC) network models have also been applied, whereby an electrical analogy is used to simplify building thermal performance and the temperature profile of individual components; however, the modelling simplifications can result in less accurate results. Data-driven methodologies, which have gained much interest lately for examining building energy systems [17], are surrogate models produced from actual data or synthetic profiles, and they can estimate a building's energy use and/or interior temperature more quickly than earlier techniques. From a power system perspective, and/or implementation of demand response programmes, the thermal performance of each building is generally not seen as being that critical, and, instead, focus is placed on the aggregated electrical power consumption of multiple loads, and simplified models are employed to reduce computational time. However, a model which could predict the energy consumption of individual buildings, while providing reasonably fast computation time, such that a sufficiently large and diverse fleet of buildings could be simulated and aggregated to system level would be preferable. Given their ability to tackle non-linear and complex patterns, while still achieving predictions with high accuracy, machine learning (ML)-based methods, as part of data-driven approaches, offer a potential solution for energy prediction in complex energy systems, which makes them superior to other solutions. However, availability of data can often present an implementation challenge for such methods. The concept of surrogate data centre models for predicting the thermal performance, server temperature and energy usage of data centre using machine learning techniques has not previously been investigated.
ML models require historical training data, and here, synthetic data from different scenarios is leveraged for emulator training. The energy performance of data centres is formulated as a regression ML problem for two data centre types, with discussion on the main technical challenges for flexibility provision from data centres, and their potential impact on power system operation. The dynamic data centre building simulation employed here is merely a tool to support investigation of the feasibility of a system-wide fleet of data centres implementing a demand response programme.
2) Integration of ML Within Optimisation Framework: AI and ML algorithms have attracted interest from research and industrial sectors for providing insightful and sustainable decisions. To date, focus has been placed on classical optimisation frameworks, such as mixed-integer linear programming (MILP) [10], or AI-based decision making methodologies, such as "reinforcement learning" [17], in which ML/deep learning methods are often used to define system components. Reinforcement learning requires knowledge of the system state and understanding of the impacts of individual choices on overall system performance, which can be impractical or computationally costly, particularly for power systems, which are naturally Fig. 1. Model overview, incorporating physics-based data centre models, representative ML models for a system-wide fleet of data centres, and a power system network, for optimal data centre load shifting based on power system CO 2 intensity. complex and uncertain [20]. Integrating ML algorithms into classical optimisation frameworks represents an alternative method [21]. However, executing multi-period optimisation has not previously been considered, including incorporating an ML end use model within a typical optimisation framework.
The main contributions of the current work can be outlined as: 1) dynamic modelling of data centres, with the aim of assessing their energy performance, before generating a synthetic ML training dataset to efficiently predict their electricity consumption and server temperatures, 2) investigating distinct flexibility solutions for data centres in terms of IT workload rescheduling, cooling equipment, and battery utilisation, 3) bridging ML model and optimisation frameworks for time-shifting of data centre demand based on (system) carbon intensity, and 4) studying the impact of carbon-aware load shifting from a power system perspective. The remaining paper is organised as follows: Section II presents an overview of the modelling structure. Section III presents the data centre dynamic modelling using IES-VE software, and introduces the machine learning models. The optimisation framework, including data centre flexibility, is presented in Section IV, and simulation results for the considered test system with different flexibility measures are illustrated in Section V. Section VI concludes the paper.

II. MODEL OVERVIEW
Data centres are viewed here as flexible loads, such that their electricity consumption can be partially shifted away from high intensity CO 2 periods towards low intensity periods, based upon artificial intelligence and optimisation techniques, as shown in Fig. 1. At the top level, an actual data centre. with two types of cooling systems, is modelled and simulated using IES-VE, a dynamic building simulation tool [22] to determine data centre power consumption and server temperature behaviours. In the middle level, an application programming interface (API), developed in Python, scrapes actual IT workloads from available online platforms, such as Google, to refine the IES-VE simulation model parameters. Subsequently, synthetic datasets (for each cooling type) are formed from the IES-VE outputs, based on simulating year-long variations in IT workloads and server temperature setpoints against time-varying meteorological data.
ML models are subsequently trained to predict data centre power consumption and server temperatures, depending on the type of cooling, and with an aggregated system-wide fleet of data centres obtained through combining differing numbers of ML models for each cooling type, with randomised parameters (IT load and temperature setpoints). Finally, in the bottom layer, data centres are supplied with forecasts of (power system) carbon intensity by the TSO, which cause them to adjust (time shift) their load consumption patterns. Individual data centres are connected to different electrical network buses, with different IT workloads and temperature setpoints. In addition, an API, "Smart Grid Dashboard, EirGrid" [23], collects real-time/historical information relating to system demand, wind power, and carbon intensity for the Irish grid. Subsequently, a multi-period optimal DC power flow (OPF) optimisation is solved, incorporating data centres with the ability to shift their loads based on the carbon intensity input.

A. IT Workload
Data centre workload is defined by the volume of arriving jobs submitted by users, with each job assigned to a specific tier [8], such that higher tier jobs permit almost zero tolerance to disruptions and must be completed immediately upon arrival, while lower tier jobs are more delay tolerant and can be completed any time before their deadline. Here, jobs are divided into two categories, namely inflexible (interactive or real-time) or flexible (delay tolerant or batch) jobs [24]. Examples of flexible IT workloads include data compaction and distributed computation for processing videos. In contrast, inflexible IT workloads include streaming of videos on Netflix, YouTube, tweeting on Twitter or posting a photo on Instagram, which represent tasks that must be performed immediately.
Considering the impact of realistic IT workloads, comprising both flexible and inflexible categories, on data centre energy consumption and internal server temperatures, is often neglected [10], [11]. To this end, historical data [25] for the year 2020 in Ireland is normalised here to reflect the national inflexible IT workload, accounting for, for example, utilisation of Gmail, YouTube, Blogger, Websearch, Google Sites, and Google Docs. Regarding the flexible IT workload, a Microsoft Azure Cloud Computing Service dataset for a large company in Ireland has been anonymised. The flexible IT workload data has, subsequently, been randomised to capture a diverse fleet of commercial companies using cloud computing services. The aggregated approach tends to reduce workload forecast uncertainty. Normalised data for the inflexible dataset is illustrated in Fig. 2(a) for one week for various categories, while Fig. 2(b) shows the aggregated inflexible and flexible IT workload. As per [24], it is assumed that the inflexible and flexible workloads are roughly equal.

B. Data Centre Dynamic Model
In order to study the energy performance of data centres, the IES-VE commercial software [22] is used, which incorporates the dynamic thermal physics of data centres. After an initial data  gathering campaign, the data centre model is developed using the building dynamic simulation software IES-VE. The simulated facility is a real data centre of approx. 40,000 m 2 with a total volume of approx. 330,000 m 3 . The building consists of two main parts: cell rooms, where servers and IT equipment are located (approx. 19,000 m 2 ) and auxiliary facilities (remaining 21,000 m 2 ). The cells consist of 24 different units, approx. 800 m 2 each (see Fig. 3), while the auxiliary rooms and facilities include: administrative rooms, offices, backup generation units, electrical units, mechanical and storage rooms, UPS units, amenities and personnel facilities. Each cell unit is modelled assuming a 60 cm steel framing structure for the external walls, with a total U value of 0.22, lighting internal gains modelled considering fluorescent lighting with 7 W/m 2 intensity, and IT equipment and servers modelled with 2500 W/m 2 intensity.
Two cooling strategies, "standard" and "free," are considered with further details provided later in Section III-B1 and III-B2. Generally speaking, for standard cooling, each cell unit is considered as a single thermal zone served by a dedicated air-side heating, ventilation, and air conditioning (HVAC) system. The HVAC is modelled as a packaged single-zone computer room air-handling (CRAH) unit, consisting of a chilled-water cooling coil (no heat), a variable volume fan, and relative humidity zone control. For the free cooling configuration, a similar HVAC system is considered with the addition of an outside air economizer with a dry-bulb temperature high limit. The chilled water loop was modelled considering electric water-cooled chillers with variable speed pumps on the secondary circuit with a total installed cooling capacity of 90 MW. The model considers outdoor temperature, external relative humidity, and IT workloads as input files, before determining the internal server temperatures, average building temperature, cooling system controls and equipment, and server energy consumption. In this study, simulations are performed for a one year period with a time step resolution of 30 minutes. Due to the complexity of the physical model, a year-long simulation for a single data centre takes approx. 4 hours, with standard cooling, and free cooling models running separately on a PC with Core i7-8750H, CPU @ 2.20 GHz, and 16 GB of RAM.
1) Standard Cooling Model: A simplified configuration for a data centre with standard cooling is shown in Fig. 4(a), including backup generators and batteries in addition to the main grid supply connection. Server temperatures must be maintained within a specified range for correct equipment operation and safety reasons. Chillers are used to cool the servers, with circulating cold water (supplied by the chillers) cooling the hot air. (1) represents on/off operation of the cooling system based on the temperature setpoint, T ST , and T Server is the server temperature. T ST must remain within the range T min ≤ T ST ≤ T max to ensure that the servers operate safely and reliably, where T max/min is the 2) Free Cooling Model: In contrast to standard cooling, which is entirely dependent on chillers for cooling down the servers, free cooling can also employ outside air, with a logic controller able to switch between both options. If outside air is being used for cooling purposes, the mixer in Fig. 4(b) combines the hot air from the servers with the outside airflow to cool the servers. Otherwise, the mixer blocks external airflow and only the chillers are used for server cooling. The control logic to utilise outside air is summarised in (2), where T amb represents the outdoor temperature and δ is a safety deadband. The on/off chiller control logic is similar to (1).

3) Scenarios & Simulation Results:
In order to analyse and compare the performance of standard and free cooling, before constructing individual ML models (Section III-C), four different IES-VE scenarios have been considered, as summarised in Table I. The same outdoor temperature and external relative humidity annual profiles are applied for each scenario, as measured at a Dublin weather station for 2020, along with the same IT workload. Year-long simulations are performed assuming, (1) a fixed temperature setpoint of 26°C and (2) a randomised temperature setpoint with a mean of 26°C and variance, δ 2 , of 0.25, as outlined in Table I. In reality, a random temperature variation is not realistic, but it is applied here to support training of the machine learning model. A training set consisting of 4 × 8760 × 2 = 70, 080 rows of data is subsequently generated at a 30-minute resolution.
For the same operating conditions, and in an Ireland context, free cooling requires approx. 11% less electricity annually, due to the available opportunities for outside air cooling. However, standard cooling enables tighter control of the server temperatures, most noticeably when IT workloads are higher. Fig. 5(a) shows a comparison of the power consumption from both free and standard cooling arrangements for a one week winter period, with an average outdoor temperature of 7.9°C and an average external relative humidity of 89%, considering only inflexible IT workloads. It can be seen that the IT workload is the main driver for power consumption, but, as expected, free cooling helps to reduce dependency on the chillers. For the time period shown in Fig. 5(a), data centres with free cooling consume an average/maximum of 20.5/24.6 MW, which increases to 23.0/31.4 MW with standard cooling. Fig. 5(b) shows a pie chart  of energy consumption for the same period, showing that chillers account for 3.7% using free cooling, but 12.1% with standard cooling.

C. Machine Learning
Machine learning algorithms are now introduced with the aim of predicting data centre power consumption and server temperatures.
Typical load and temperature forecasting features are incorporated to predict the target variables of interest, i.e. power consumption and server temperature, as outlined in Table II . IT workload is considered as an input variable due to its large impact on both target variables. Outdoor temperature, external relative humidity, temperature setpoint, and calendar information are also considered. Temporal and holiday features are transformed from categorical data to numerical data through a one-hot encoding procedure to capture the impact of normal working days, and otherwise. The thermal inertia of the building, and its impact on the temperature and power consumption of the servers, is recognised by including lagged values of the server temperatures for 4 hours (i. e. t − 1, t − 2, t − 3, and t − 4).
For training and testing purposes, the dataset has been split 70:30, with each data feature normalised using the MinMaxScaler approach [26], based upon the training dataset. Three distinct ML regression models have been employed, namely multiple linear regression (MLR), random forest regression (RF), and extreme gradient boosting (XGBoost) to predict the target variables of interest. The choice of algorithms is based on industry deployment and consideration of some principles: 1) complex models to be avoided, while also reducing the number of variables; 2) following the international performance measurement and verification protocol (IPMV) [27], all measurement and verification activities should be transparent and adaptable to future changes; and 3) lightweight loading on cloud system services, e.g. Microsoft Azure, to enable organisations to focus on business operations rather than complicated IT infrastructure maintenance. For reasons of transparency, interpretability, flexibility to future changes, and accessibility across all cloud system services, MLR would be anticipated to be a preferred option, subject to meeting modelling performance requirements. For similar reasons, the RF approach is also popular, and unlike other ML models, such as k-nearest neighbours, its default settings appear to work well across a wide variety of datasets [28]. XGBOOST is selected as it offers one of the fastest ML approaches with high resilience and flexibility, as seen in Kaggle ML contests [29]. In each case, 4 separate ML models are required, representing power consumption and server temperature, for standard and free cooling data centres.
Multiple Linear Regression: MLR is a well-established ML approach that aims to find the relationship between several independent variables and one dependent variable, (3), where, for each observation, Y is the dependent variable, X k is a vector of independent variables, and β 0 , β k , and ε represent a constant, slope coefficients, and the model error. MLR can be understood, in an ML context, by considering input X as the training dataset and input Y as the target variable, with β k to be suitably selected to represent the influence of the considered features in the dataset [30]. Random Forest Regressor: Random forest proceeds in an iterative manner to perform binary feature splits, and, in so doing, create a decision tree structure. By randomly selecting feature subsets, rather than investigating all possible splits, individual nodes are split based on the "best" feature, until further splits are no longer possible. RF employs a minimum number of "leaves" in order to reduce the likelihood of problem overfitting [28].
Gradient Boosting Regressor: Finally, gradient boosting (GB) attempts to achieve a "good" prediction from an ensemble of "poor" prediction models, which is achieved by combining "poor learners," based on the prediction errors, to create a "good learner". In particular, XGBRegressor (XGBR) is an efficient open-source representation of the GB algorithm, capable of achieving high computational performance [29]. Table III summarises a performance evaluation for the considered ML models, where common forecasting evaluation metrics are considered, including mean absolute error (MAE), root mean squared error (RMSE), mean absolute percentage error (MAPE) and R-squared value. In general, it tends to be easier to predict the server temperature for the standard cooling over the free cooling model, while the reverse is true for the power consumption, which follows from the nature of the cooling mechanism in each case. A range of approaches are available to improve performance, with hyperparameter tuning playing a vital role. A hyperparameter is a parameter whose value is established prior to the learning process. A number of tuning methods, such as grid search [30], are available, while FLAML [31] has been used here to optimise the hyperparameter values, e.g. number of estimators, maximum number of leaves. Given the similar performance of the three ML approaches, the MLR approach is ultimately chosen as it avoids additional non-linearities and complexities as part of the later optimisation problem (Section III). Fig. 6 shows predictions of the server temperature and power consumption using standard cooling, based on the inputs defined in Table II. Considering the MLR coefficients, the predicted server temperatures for the free cooling models tend to be much more sensitive to IT workload compared to the standard cooling models (∼ 45%), since the server temperature is quite dependent on previous hours. In contrast, the power consumption in standard cooling models is more influenced by the IT workload (18%) relative to the free cooling models.
So far, ML models of individual data centres have been developed to predict electrical demand and server temperatures. Subsequently, an aggregated system-wide flexible load is estimated through simulating individual stochastic devices, formed from differing numbers of data centre types (standard or free cooling), with randomised parameters (IT load and temperature setpoint). The aggregated ML approach adopted provides an opportunity to obtain insight over critical components of each data centre within the considered system-wide fleet from a high-level perspective, while ensuring fast, yet accurate, prediction of their behaviour.

A. Power System Scheduling
The optimisation framework involves a multi-period optimal power flow, which aims to minimise the total operating (fuel) cost, subject to technical constraints for generating units, battery storage, system non-synchronous share [10], and power flow network limits. Additional constraints are introduced in Section IV-B to capture carbon-aware scheduling of the data centres, which bridges to the ML model, Section III-C. The optimisation framework and DC power flow constraints are based on [32]. The objective function is defined in (4): Min.
where, C(P g,t ) is the operational fuel cost of unit g at time t, generating power P g,t . V OLL is the value of lost load (due to load shedding) at bus i, LS i,t . Technical constraints, such as maximum/minimum power and ramp up/down rates for generating units, battery energy storage systems and their associated constraints, and generation-demand balancing are also considered, as per previous studies [32].

B. Assessment of Carbon-Aware Flexibility Measures
Given the commitment of major technology companies towards achieving sustainable development goals and reducing carbon emissions, an understanding of flexibility measures that can be provided by data centres, as an increasingly significant load sector, is required. Furthermore, the EU emissions trading system provides an additional motivation for companies to increase their flexibility. Consequently, various technological options are now investigated, based on following a carbon intensity signal, to increase the operational flexibility available from data centres. As stated in Section III-C, multiple linear regression is adopted here to predict the power consumption and server temperature of the data centre population. Data centres are typically connected to the power system via a number of power distribution units (PDUs) on medium-voltage feeders. The PDUs support IT and cooling equipment, with each PDU supplying a few thousand machines, within an overall cluster. An operating system handles IT jobs within a cluster, and assigns tasks across available machines, in terms of allocating resources, e.g. CPU/RAM, job start times [8].
1) IT Virtual Boundary: A virtual boundary can be defined to reduce cluster-level CPU capacity (by adjusting availability for incoming flexible tasks) when (power system) carbon intensity is high, and then raising the virtual boundary (making more CPUs available) when carbon intensity is low. Consequently, flexible IT workloads can be delayed until resources become available, as demonstrated in Fig. 7, where, cis t represents the time-varying carbon intensity for hour t, being higher here during the middle of the sample day. Given an upper bound on cluster-level capacity, Y DT C,max d , for a given day, d ∈ D, the (initial) flexible IT workload is subsequently reduced beneath the imposed virtual boundary, based upon a time-varying coefficient δ t . The coefficient is inversely related to the system carbon intensity, i.e. δ t ∼ k/cis t , where k is a constant. If the IT workload, IT DT C t , in each data centre, DT C, accounts for both flexible and inflexible categories at time t, (5) forces the flexible IT workloads to be shifted towards low carbon intensity periods, subject to all tasks being completed by the end of the day, (6).
The power consumption, P DT C t , and equivalent server temperature, T server t , are calculated using the MLP approach for each data centre server. Since changes in IT workload can significantly affect the server temperatures, minimum/maximum thresholds are imposed, T DT C,min,server and T DT C,max,server . Similarly, data centre power consumption is limited to P DT C,max , and the chiller setpoints are maintained within minimum/maximum boundaries, T DT C,min,ST and T DT C,max,ST , to ensure safe operation.
2) Flexibility From Chillers: Given that the temperature setpoint, T ST , affects the power consumption and server temperature, adjusting the max/min temperature boundaries, based on carbon intensity, can be applied, whereby higher temperatures are permitted during periods of high carbon intensity (with a consequent reduction in power consumption), subject to server safety limits.
3) On-Site Generation: Since most data centres include some form of on-site generation, if only with limited storage capacity, individual data centres could alternatively be disconnected from the main grid, and operated with on-site battery storage/UPS, or even renewables.

V. CASE STUDY
The impact of data centre carbon aware scheduling on power system operation is now studied, based on a modified and simplified model of the Irish power system, including the Republic of Ireland and Northern Ireland [33], which consists of 25 conventional units, 11 (aggregated) wind farm clusters, and three battery energy storage systems. A Python-based energy API scrapes data from the EirGrid smart grid dashboard [23], to obtain real-time/historical data regarding system demand, wind generation, and CO 2 intensity. A total of 10 data centres is assumed, each connected to a specific bus, accounting for ∼ 11% of the total electrical demand. Since IT workload is the main driver for data centre power consumption, a national TSO-level perspective is assumed. IT workloads and temperature setpoints are slightly randomised to capture the variability associated with individual data centres, while hourly profiles for the outside temperature and relative humidity for Dublin are obtained from Met Éireann. Finally, the optimisation problem is coded in Python and solved via Pyomo.
Six scenarios are defined, as part of assessing the potential flexibility from a system-wide fleet of data centres: (1) a "Base" scenario assumes that data centre power consumption is fixed and inflexible across the entire year, (2) '100% Flex" permits the entire data centre load to be flexible, subject to the total aggregated load being unchanged at the end of each day, (3) "FlexIT-NoTemp" is similar to [8], whereby flexible IT loads Fig. 8. System electrical demand, before and after applying carbon aware signals to data centres, a winter day. can be shifted/re-arranged within each 24 hr period, but server temperature constraints are not imposed, (4) "FlexIT+Temp" is similar to (3), except that server temperatures must be restricted within safety margins, where T min,server = 15°C, T max,server = 32°C, (5) "FlexChiller" assumes that the IT workload is fixed, but chiller temperature setpoints can be adjusted based on carbon intensity, as described in Section IV-B2, and finally (6) "FlexChiller+FlexIT" is analogous to (4), but with the additional ability to adjust chiller temperature setpoints. Scenario (4) differs from scenarios (5) and (6) in that the temperature setpoint can fluctuate between fixed boundaries, while (5) and (6) enable the setpoint boundaries to vary based on the carbon intensity.
A representative winter (January) day is now considered, with summary optimised scheduling results, based on the six flexibility scenarios, presented in Fig. 8 and Table IV. For scenarios (2)-(4), the IT virtual capacity of each data centre is adjusted for each timestep based on the estimated CO 2 intensity. Fig. 8 shows the system demand, with load reductions seen from high intensity CO 2 periods towards low intensity periods, relative to the "Base" scenario. For the given day, the highest CO 2 period occurs between 9 am to 5 pm, leading to load reductions of 6.5%, 8.3%, 4.5%, 0.5%, and 5.5% for the "100% Flex," "FlexIT-NoTemp," "FlexIT+Temp," "FlexChiller" and "FlexChiller+FlexIT" scenarios, respectively. For "100% Flex," the daily power consumption is unchanged, based on how the scenario is defined. However, for "FlexIT-NoTemp" and "FlexIT+Temp," daily load reductions of 2.8% and 0.5% are achieved, due to re-scheduling IT workloads towards times when the cooling requirement is reduced (lower IT loads and lower ambient temperatures), particularly for "FlexIT-NoTemp," given that server temperatures do not restrict the ability to reschedule the IT workload. Similarly, 0.6% and 0.3% load reductions are observed by adjusting the chiller temperature setpoints. On average, a 5.2% operating cost reduction is achieved due to the additional flexibility provided by the data centres compared to the "Base" scenario.
Comparing the various carbon aware approaches against the "Base" scenario, it can be seen that the load profile becomes somewhat flatter, achieving standard deviation reductions of 12.2%, 33.2%, 22.5%, 6.4%, and 29.1% for the '100% Flex," "FlexIT-NoTemp," "FlexIT+Temp," "FlexChiller" and "Flex-Chiller+FlexIT" scenarios. The reduced variability also implies that the ramping requirements for online (conventional) generators are noticeably reduced. "FlexIT-NoTemp" is most effective at reducing the total operational cost, and flattening the load profile, with "FlexChiller+FlexIT" next in line, but, by scenario design, excessive variations in server temperature are not considered.
For the same winter day, Fig. 9 and Fig. 10 show the flexible IT workload and server temperatures for the "FlexIT-NoTemp" and "FlexIT+Temp" scenarios. Data centres 1-5 employ standard cooling, while 6-10 utilise free cooling. In both cases, the IT workload is shifted from the middle of the day to early morning or late night periods, avoiding the carbon intense periods. As seen in Fig. 9(b), server temperatures are kept well below 32°C. Not surprisingly, the free cooling data centres reach higher server temperatures, while standard cooling achieves lower temperatures, due to 24/7 operation of their cooling systems. Fig. 10(a) shows that IT workloads are significantly shifted from the middle of the day, but for a few data centres, server temperatures approach 150°C since temperature constraints do not apply, which is, of course, not acceptable. However, bounds on server temperatures, as outlined in Fig. 9(b) will restrict the ability to shift load at certain times, which will further depend on the nature of the cooling (standard or free) employed by individual data centres, and the external power system / time of year conditions on ambient temperature, IT workload, etc.

A. Modelling Complexity & Results Comparison
A range of different modelling and analysis approaches have previously been proposed for assessing the available flexibility from data centres. The present work is compared here against four alternative approaches [8], [10], [11], [13], and five indices are considered relating to modelling complexity, as summarised in Table V. The present research is the only option that applies a data-driven method, taking into account not only physical dynamic models of data centres but also ML techniques for reducing the computational time. Furthermore, for system-wide aggregation, a bottom-up strategy is implemented here, whereby a cluster of data centres is represented as multiple ML models with stochastic parameters. Although [10], [11] study a systemwide fleet of data centres, it is fairly simply assumed that data centres account for a given percentage of overall grid demand, and specific data centre models are not included. In contrast, models of individual data centres are the focus of [8], [13].
In terms of measures for adjusting the power consumption of data centres, each study is different: for example, the present study and [8] propose avoiding consumption during carbon intensive periods, while [10], [11] focus on how data centres can support power systems with high renewable shares. The results presented here align with those provided in [10], in terms of data centres supporting system-level flexibility needs, and how IT workload rescheduling can reinforce available flexibility [8]. However, notably, the present study also considers how restrictions on server temperature (and other) variations limit the available flexibility at certain times, while measures such as adjusting chiller temperature setpoints are shown to improve the available flexibility.

B. Replicability, Scalability, and Generalisability
The generalisability of the proposed model is not affected by the modelling approach applied in this study. Fundamentally, its generalisability and scalability are constrained by data availability, which fast digitalisation of the industry is helping to avoid. In addition, the integrated modelling approach adopted here has limited weighting placed on building archetypes, as the main objective was to create a system-wide heterogeneous fleet of data centres by considering 1) different types of cooling systems, 2) distinct weather conditions, in terms of temperature and relative humidity, and 3) randomised IT workloads. Indeed, the uncertainty associated with modelling multiple end uses can be mitigated by randomising the primary inputs and associated parameters. Representing the data centre response in the form of a large and diverse fleet helps to reduce the uncertainty surrounding the available demand response [34].

VI. CONCLUSION
The potential flexibility from data centres, in order to shift their energy consumption based on a carbon intensity signal is presented. Using detailed thermodynamic simulations of free cooling and standard cooling data centres, a representative dataset of data centre dynamic behaviour is created. Machine learning models are subsequently designed and trained to predict the power consumption and server temperature of individual (free or standard cooling) data centres. A fleet of data centres is represented by a suite of ML models, which is then integrated within a power system unit scheduling framework. Five flexibility scenarios are introduced and compared, with the aim of determining the achievable flexibility from a system-wide portfolio of data centres, focusing on concerns relating to server temperatures. The consequent impacts on the system demand load profile are assessed, including reduced flexibility requirements, and maximum upward ramp rates.
As part of future work, it is intended to consider how best to define carbon intensity and carbon footprint for a system, and specifically how the network location of data centres should play into reducing the observed carbon intensity. A much wider range of system conditions should also be considered, as part of judging the cost effectiveness and utilisation of individual implemented strategies. The work here has also focussed on demand shaping, but it is recognised that data centres can potentially provide various reserve products, including a fast frequency response, which would be particularly valuable in supporting system stability as part of the transition towards low inertia power systems.