Field Study on Actual Usage of Occupancy-Reactive Space Heating Control

We present evaluation of actual use of an occupancy-reactive space heating control, which changes set-point temperatures in space heating for energy-savings based on changes in occupancy state. We performed an experiment over two winter months in Lyon, France. In this experiment, occupants were provided with occupancy-reactive and pre-scheduled controls via a home energy management system (HEMS) and they were also allowed to control space heating manually via a thermostat or the HEMS. Occupants decided which control to use from among the two advanced controls and manual control, whereas control availability was limited in some experimental periods. To grasp actual usage of the two advanced controls, we introduce energy-saving potential for valve-regulated space heating and determine numbers of frequent users who applied any of the occupancy-reactive, the pre-scheduled or manual control. We also analyze actual energy consumption of space heating of the frequent users of each control. Our findings suggest energy-saving effects by the occupancy-reactive control, but the results show that the number of the occupancy-reactive control users in the experiment was not so large. This observation encourages reconsideration of the assumption that advanced controls such as the occupancy-reactive control are used fully by occupants in previous studies, indicating a necessity for promoting comprehension and active use of occupancy-reactive controls.


I. INTRODUCTION
The 2015 Paris Agreement calls for further reduction of greenhouse gases such as carbon dioxide [1], and the International Energy Agency has pointed out that improved energy efficiency is an important aspect of such reductions [2]. Space heating consumes a large portion of energy expenditure in cold regions such as Europe, with space heating in residential and non-residential areas accounting for 70% and 40% of total energy consumption, respectively, in the EU [3]. Even worldwide, space heating consumes more energy than do other categories such as domestic hot water and cooling [4]. It is thus important to develop energy-saving methods for space heating.
Previous simulations and field studies have compared occupancy-driven HVAC controls with ''always on'' and ''pre-scheduled'' controls [5]. In such a comparison, to our knowledge, previous studies have tacitly assumed full use of occupancy-driven controls by occupants. In actual situations, occupants change set-point temperatures and select occupancy-driven or pre-scheduled controls according to their own will. Therefore, the extent to which occupants use occupancy-driven controls in actual environments and how use of such controls influences energy consumption remain unclear. The aim of this study is thus to grasp actual usage of an occupancy-driven control and its effect on energy use.
To that end, we focused on evaluating an occupancyreactive control for space heating through a field study. In this study, residents of 36 apartments in France voluntarily used a HEMS implementing the occupancy-reactive space heating control. The HEMS also provided occupants with a pre-scheduled control, which is usually implemented on a programmable or smart thermostat [33], [34]. They were also allowed to control space heating manually via a thermostat or the HEMS. This experiment was performed over two winter months. Through comparison of the occupancyreactive, pre-scheduled, and manual controls, we show how the occupancy-reactive control was used by the occupants.
References [35], [36] presented preliminary results of this experiment, showing energy-saving potential (described below) of the occupancy-reactive control in a two-week experiment included in the experiment described in this paper. This study primarily extends those studies as follows: 1) Analysis of experiment results over two months. 2) Verification of the extent to which the occupancyreactive control was used. 3) Confirmation of actual energy consumption of space heating under use of the occupancy-reactive control. 4) Comparison between the occupancy-reactive and pre-scheduled controls.
These evaluations contribute to understanding the use of occupancy-reactive controls under actual environments and clarifying remaining issues for promotion of those controls. The remainder of this paper is structured as follows. Section II briefly reviews the previous studies about the occupancy-driven controls. Section III describes the occupancy-reactive and pre-scheduled controls used in this study. Section IV describes the experiment, giving an overview of the space heating system and HEMS used in the experiment. Section V evaluates the occupancy-reactive control, and Section VI shows the evaluation results. Section VII discusses these results, and Section VIII concludes.

II. RELATED WORK
This section is devoted to a brief review of the previous studies about the occupancy-driven controls.
Since the occupancy-reactive controls are relatively easy to implement in actual situations, there are a certain number of field studies [10], [14], [16], [20], [21]. References [10], [16], [21] treated university dormitory buildings for their field study and adopted smart thermostats that implemented an occupancy-reactive control. Reference [14] used a university building for their field study and adopted a neural network based occupancy model for their occupancy-reactive control. Reference [20] used a university office to compare a rule-based occupancy-reactive control with a model predictive control (MPC) based control that also used occupancy information. As for simulation study of the occupancy-reactive control, [6] evaluated their occupancy-reactive control based on the actual occupancy data and [24] developed an occupancy simulator for including random behavior of occupancy into energy simulation.
In contrast to the occupancy-reactive controls, most of the studies as to the occupancy-predictive controls are simulation studies [7], [9], [11]- [13], [15], [17]- [19], [22], [23]. Reference [7] predicted occupant arrival time from public dataset. References [9], [11], [13], [15] used a Markov model and a machine learning method to predict occupancy. Predicted occupancy was used for pre-cooling or pre-heating a target space. References [18], [19], [22] developed MPC-based occupancy-predictive controls and considered weather forecast in their models, whereas [12] compared an MPC-based occupancy-predictive control with the rule-based occupancy-reactive control studied in [20]. Reference [17] compared occupancy prediction algorithms that were used for the same occupancy-predictive control. Reference [23] predicted occupancy by several machine learning methods for a rule-based occupancy-predictive control that considered building thermal load. Reference [8] notably carried out a filed study for their occupancy-predictive control. Five families participated in this study in winter. The experiment was performed for 61 days on average.
As described in Section I, the studies summarized here assumed full use of occupancy-driven controls by occupants. Actual use of occupancy-driven controls under the situation that occupants can decide whether to use has not been made clear by these studies.

III. SPACE HEATING CONTROL METHOD
This section describes the occupancy-reactive and prescheduled controls that were implemented in the HEMS used in the experiment.
The two control methods depend on four operational modes for space heating: ''Normal,'' ''Reduction 1'', ''Reduction 2,'' and ''Lowest temperature.'' Set-point temperatures for space heating are determined by the operational mode. Fig. 1 shows the relations between the set-point temperatures under each operational mode. The set-point temperature under the ''Normal'' mode S N is specified by an occupant. The set-point temperatures under the ''Reduction 1'' and ''Reduction 2'' modes are respectively S R 1 = S N − 2 • C and S R 2 = S N − 4 • C. The set-point temperature under the ''Lowest temperature'' mode is S L = 10 • C, effectively halting space heating since some thermostats do not have a turn-off mode.  The following two control methods assume that each room has a thermostat and is individually heated.

A. OCCUPANCY-REACTIVE CONTROL
The occupancy-reactive control described here considers the occupancy state of an apartment or house and that of each room therein. The occupancy-reactive control has three kinds of set-point temperature setbacks: • Setback 1: Revert a room set-point temperature to S R 1 if the room becomes unoccupied.
• Setback 2: Revert a room set-point temperature to S R 2 if the entire apartment becomes unoccupied.
• Setback 3: Revert a room set-point temperature to S R 1 if the room is a kitchen and the kitchen has been occupied for at least some designated time. The occupants in each apartment can select in which room Setback 1 and/or Setback 2 are activated. After Setback 1 or 2 has been executed, the set-point temperature is reset to S N , when the room or apartment again becomes occupied. Occupants can also determine whether Setback 3 is used. After Setback 3 has been executed, the set-point temperature is also reset to S N , when the kitchen becomes unoccupied. Fig. 2 shows a schematic of the three kinds of setbacks.
The design concept is that longer vacancy sets lower set-point for energy-saving. Since absence of an apartment may take longer time than vacancy of a room, we use S R 1 for room vacancy and S R 2 for apartment absence (S R 1 > S R 2 ). Moreover, Setback 2 can start after Setback 1 operates, if occupants are unoccupied in a room and then leave the apartment. Setback 3 was designed for reverting set-point during cooking, by expecting occupants to stay at a kitchen for a long time for cooking and feel a little hot during cooking. We used S R 1 for Setback 3, not S R 2 , because we intended to mitigate a little hot brought about by cooking.
Since the experiment was devoted to a research project (see Section IV-A), we tried to apply Setback 3, in addition to usual setbacks during vacancy such as Setbacks 1 and 2. The differences between S N and S R 1 or S R 2 were determined by discussion on the authors and the French team engaged in the experiment.

B. PRE-SCHEDULED CONTROL
The pre-scheduled control described here changes set-point temperatures according to a schedule input by occupants. Fig. 3 shows a schematic of the pre-scheduled control, in which the set-point temperature is specified at thirty-minute intervals. Occupants can set a room set-point temperature from among S N , S R 1 , or S R 2 . Setbacks under the occupancy-reactive control. The occupied time in the bottom rhombus means time during which the kitchen has been occupied. After Setback 1 or 2 has been executed, the set-point temperature is reset to S N , when the room or apartment again becomes occupied. After Setback 3 has been executed, the set-point temperature is also reset to S N , when the kitchen becomes unoccupied. As in the case of the occupancy-reactive control, the pre-scheduled control changes the set-point temperature of each room in an apartment. Occupants can select which room is under pre-scheduled control.

C. ADDITIONAL CONTROL
The HEMS in this study uses the window open/close state to include another space heating control besides occupancy-reactive and pre-scheduled controls. That control reverts the set-point temperature of a room to S L if a room window is opened, and the set-point temperature is reset to S N if the window is closed again. Evaluation of this control is also described below.

A. ENVIRONMENT: SPACE HEATING SYSTEM AND HEMS
This subsection describes the experimental environment. The same environment was used in [37] and a similar description of the environment was given in that reference.
The experiment, one research topic in a French-Japanese joint research project, was carried out in all rooms of 36 apartments in a newly constructed building in a residential area in Lyon, France. Table 1 shows the number of apartments by room layout and the areas of each apartment. The apartments have eight room layouts.
Space heating in the apartments used hot water produced by a 600-kW gas boiler and 98-kW combined heat and power (CHP) in the basement of the building. Fig. 4 shows the space heating system and HEMS in an apartment. Each apartment had an energy meter for hot water used for space heating. Each room in an apartment had a thermostat and hot-water radiator. Apartments also had a hot water distribution panel (Oventrop Multidis SF [38]), each valve in which was connected to a valve actuator for adjusting valve opening values and regulating water flow volume. The valve actuator receives a valve opening value from the thermostat, which calculates that value based on the set-point temperature input to the thermostat and the room temperature as measured by the thermostat.
A HEMS was installed into each apartment, which had a HEMS box as a gateway and local controller. The HEMS box was connected to the thermostats and valve actuators in the apartment through a KNX network. (KNX is a widely used building protocol in Europe.) The HEMX box collected historical data of set-point temperatures and valve opening values from the thermostats. These data were collected at one-minute intervals.
The control methods described in Section III were implemented in the HEMS box. The HEMS box recorded execution time of the occupancy-reactive control including each setback, the pre-scheduled control, and the additional control. The occupants in the apartment could select which control method was applied to a room, except for the window-state control described in Section III-C, which was assigned to mandatory control in response to a request by the French team. Occupants could also change the normal-mode set-point temperature S N and the thermostat mode manually via the thermostat or the HEMS.
The HEMS box estimated occupancy information in rooms and apartments for the occupancy-reactive control by using data from motion sensors installed in each room [39]. We carried out a preliminary experiment at the phase of on-site commissioning. In this experiment, three persons stayed at an apartment and moved around the apartment more than three hours. The estimation accuracy of the occupancy estimation was 93%. As for privacy, since the used motion sensors were usual PIR motion sensors, we collected only motion information from which individuals were not identified. We and the French team of the research project explained installation of the motion sensors to the occupants, before they started to live in the building. Therefore, the occupants understood data collection from the motion sensors.

B. CONTROL METHOD ALLOCATION
The experiment consisted of four periods in which the control methods were allocated to the occupants. (See Fig. 5 for the experiment procedure and main evaluation items treated in this paper.) • Period 1 (12/19/2016-1/8/2017): Occupants were allowed to use all control methods with no restrictions.
• Period 2 (1/9/2017-1/22/2017): Occupants were randomly divided into two groups, one allowed to use the occupancy-reactive control and the other not allowed to use that control. Neither group was allowed to use pre-scheduled control.
• Period 3 (1/23/2017-2/5/2017): Occupants were again randomly divided into two groups, one allowed to use only the occupancy-reactive control and the other allowed to use only the pre-scheduled control.
• Period 4 (2/6/2017-2/26/2017): Occupants were allowed to use all control methods, as in the first period. Occupants could change set-point temperature and thermostat modes in their apartments over all four periods. Note that two groups of Period 2 were different from those of Period 3 because of random grouping of the occupants.
Although the control methods were allocated among the apartments, their use was according to occupants' will. Therefore, some of those allowed to use a given control method hardly used it in the second and third periods. Leaving use of a control method to occupants is a notable characteristic of this experiment.
As mentioned before, there are another research item that used the same environment, especially in apartment living rooms [37]. The fourth week of the experiment described in [37] was included in the first period in this experiment. However, occupants experienced no interference in that VOLUME 9, 2021 fourth week, so that no aftereffects likely occurred in the first period of this experiment.
We briefed occupants about the HEMS and its control methods when they started living in the apartments. In-person guidance about the HEMS was also frequently performed at occupant request. We performed another briefing about the experiment before it started. We also sent a brochure describing the experiment to all apartments, as described in [37]. Therefore, we assumed that occupants understood the HEMS and actively used the control methods during the experiment. 1 After the experiment, we distributed a questionnaire to all apartments to verify how occupants used the control methods and the HEMS.
Note that here we describe a field study in which the control methods were actually used by occupants under circumstances where space heating was regulated by a hot water valve, while [40], [41] described simulations of valve-regulated space heating.

V. EVALUATION
We analyzed the following evaluation items from the data obtained in this experiment. 1 This study is concerned with human research in that the occupants used space heating control. Although we provided the occupants with the control methods and control availability was limited in Periods 2 and 3, we did not force the occupants to use the control methods, and therefore, the occupants did not receive physical and psychological burden. In addition, we did not collect any biological data and sample from the occupants. Accordingly, there was no need to obtain an approval by an ethical review board. Moreover, the French team (the seller of the apartments) explained installation of the HEMS and the experiment to the occupants, when they made a contract. As above-mentioned, we fully explained the experiment to the occupants. Opt-out of the experiment was also possible before and during the experiment, but no occupants asked us for opt-out. Effectively, we carried out actions that might be required by an ethical board.

A. SELECTION OF APARTMENTS BASED ON USED SET-POINT TEMPERATURE
Some apartments used nearly no space heating throughout the experimental periods, and inclusion of those apartments affects performance evaluation of the control methods. This study therefore verified relations between the used set-point temperatures and normal-mode set-point temperatures, excluding those apartments that seemed not to use space heating during the experiment.

B. ENERGY-SAVING POTENTIAL AND USE RATE
As described in Section IV-A, it was difficult to confirm energy consumed by each of the hot-water radiators, because they had no attached energy meters. We therefore introduce energy-saving potential by a setback or by a control method for hot water radiators from their valve-opening values. The energy-saving potential described below assumes that energy use by a hot-water radiator is proportional to its valve opening. This assumption is valid for equal-percentage valves [42], which have similar valve characteristics of the hot-water distributor used in this experiment [38].
For hot-water radiators, the energy-saving potential by a setback that changes the normal-mode set-point temperature S N to a reduced one S R is where V S N and OT S N are the average valve-opening value and operating time during use of S N , respectively. In addition, V S R and OT S R are the average valve-opening value and operating time during use of S R by the setback, respectively. The average valve-opening value V S R and operating time OT S R are determined from the execution time of the setback that was recorded by the HEMS. Fig. 6 shows a schematic of the 47208 VOLUME 9, 2021 energy-saving potential by a setback that changes S N to S R . The numerator of the energy-saving potential is the product of the operating time OT S R and the difference in valve opening between the case where the set-point temperature is maintained as S N and the case where the set-point temperature is changed from S N to S R . The energy-saving potential is finally obtained as the ratio of that product to the product of valve opening and operating time in the case where the set-point temperature is maintained as S N . Note that the energy-saving potential E SB is the percentage of energy use theoretically saved by the setback to the hypothetical energy use without the setback. In the same manner as a single setback, the energy saving potential by either the occupancy-reactive control or the prescheduled control for a hot-water radiator is where R is a set of setbacks that change the set-point temperature to a reduced one. The energy-saving potential E CT is the percentage of energy use theoretically saved by the control method to the hypothetical energy use without that control. For the occupancy-reactive control, R consists of Setback 1, Setback 2, and Setback 3. Each term in (2) is a contribution of the setback that changes the set-point temperature from S N to S R . Therefore, the total energy-saving potential of a control method (the occupancy-reactive control or pre-scheduled control) can be divided into the energy-saving potential by each setback. The use rate of a control method is the extent to which that control method was applied to a hot-water radiator. The use rate of the control method is where OT CT is the operating time under the control method and OT Total is the total operating time of the hot-water radiator. For example, assuming that a control method is composed of Setbacks 1, 2, and 3, OT CT indicates the time during use of the three setbacks.
In the next section, we present the relationship between the energy-saving potential and use rate under the two control methods. The energy saving potential and use rate of an apartment under a control method are given by the averages over all hot-water radiators. The results clarify the extent to which occupants used those control methods and determine the frequently used control method for each apartment. We also compared the three setbacks of the occupancy-reactive control and the additional window open-state control. This comparison shows which setback is important with respect to energy-saving potential.

C. ACTUAL ENERGY CONSUMPTION
As described in Section IV-A, we measured the total energy consumption of space heating for an apartment, calculating energy consumption of space heating per area based on Table 1. To confirm energy-saving effects by the control methods, we determined relations among the actual energy consumption per area, energy-saving potential, and use rate. We also separately present distributions of actual energy consumption per area for apartments that frequently used the control methods and those that did not (only manual control).

D. ANALYSIS OF RESPONSES TO QUESTIONNAIRE
In the questionnaire distributed after the experiment, we asked the following questions concerning used control methods and comfort during the winter: 1) Which control method did you often use? The occupancy-reactive control, the pre-scheduled control, or only manual control? 2) How did you feel under space heating in the winter?
Comfortable or cold? We analyze the relationship between the frequently used control method answered from the occupants with that determined from the energy-saving potential (2). We also check comfort answered from the occupants based on classification by the frequently used control method determined from the energy-saving potential. Fig. 7 shows relations between used set-point temperatures and normal-mode set-point temperatures of the apartments for the four periods. Points in Fig. 7 denote averages of the used set-point temperatures and normal-mode set-point temperatures in each period. As Fig. 7 shows, the used set-point temperatures in some apartments were below 13 • C, in contrast to the fact that most normal-mode set-point temperatures were higher than 17.5 • C. We regarded such apartments as those not using space heating. Apartments using set-point temperatures below 13 • C in each of the four periods were thus omitted in the following results.  control for the four periods, which indicates a positive correlation between the two. Although it is difficult to discern the difference, a large portion (about 64%) of the points in that figure are positioned at (0, 0), indicating that many of the apartments did not use the occupancy-reactive control. There is a point whose use rate is about 100% but energy-saving potential is 0% in Fig. 8. This point is an outlier. Since only one thermostat in an apartment in the fourth period was used for a short time and the data of the other thermostats was lacking, the use rate and energy-saving potential of the apartment were coincidently calculated as 100% and 0%, respectively.

B. USAGE OF CONTROL METHOD
To investigate usage of both the occupancy-reactive and pre-scheduled controls in more detail, we confirmed the energy-saving potential of the apartments in each period, considering apartments whose energy-saving potential of a control method exceeded a threshold as control users. Table 2 shows numbers of apartments whose energy-saving potential of either the occupancy-reactive or pre-scheduled control exceeded 1%, 3%, and 5%. These apartments were regarded as control method users. The ''Only manual'' column shows the number of apartments using neither the occupancy-reactive control nor the pre-scheduled control. Those apartments were thus manual control users. Note that the pre-scheduled control was not allowed during the second period, so no apartments used that control during that period.
As Table 2 shows, at most 10 apartments used the occupancy-reactive control, and at most 6 used the pre-scheduled control. According to the 1% threshold in Table 2, the number of the apartments that used the occupancy-reactive control at least once in the experiment was 14 (that number was 19 if the apartments that used the pre-scheduled control were included). As the number of the apartments used neither the occupancy-reactive nor the pre-scheduled control throughout the experiment was also 14,  shows values for the occupancy-reactive control, ''PS'' shows values for the pre-scheduled control, and ''Only manual'' shows values for neither of the two controls. An apartment used both the control methods. That apartment was counted as both the occupancy-reactive and pre-scheduled control user. The number of apartments using the pre-scheduled control includes the apartment that also used the occupancy-reactive control, as denoted by an asterisk.
the number of the apartments that used only manual control throughout the experiment is less than that shown in Table 2. As mentioned above, use of the control method was according to occupant will, whereas the available control methods were limited in the second and third periods. The results indicate that the occupants did not necessarily actively use either the pre-scheduled control or the occupancy-reactive control. Such results regarding occupancy-reactive control were not reported in the previous study [5]. Fig. 9 shows the relationship among actual energy consumption per area of space heating, energy-saving potential, and 47210 VOLUME 9, 2021 FIGURE 9. Energy-saving potential, use rate, and actual energy consumption per area of space heating of the apartments that used the occupancy-reactive or pre-scheduled control. Here, ''OR'' and ''PS'' indicate the occupancy-reactive and pre-scheduled controls, respectively. Regression lines are added for each control.

C. INFLUENCE OF CONTROL METHOD ON ACTUAL ENERGY CONSUMPTION OF SPACE HEATING
use rate, based on frequent users of the occupancy-reactive and pre-scheduled controls. Frequent users of each control were determined as those where the energy-saving potential of each control method was 1% or more, as shown in Table 2. An apartment that used both the control methods in that table was classified as an occupancy-reactive control user, because the energy-saving potential of the occupancy-reactive control was 57% and that of the pre-scheduled control was 3%. Fig. 9(A) shows the relation between actual energy consumption and energy-saving potential, and Fig. 9(B) shows the relation between actual energy consumption and use rate. Fig. 9 suggests a negative correlation between the actual energy consumption and energy-saving potential or use rate under each control. The correlation coefficients for the occupancy-reactive and pre-scheduled controls in Fig. 9(A) were −0.20 and −0.62, respectively, and those in Fig. 9(B) were −0.16 and −0.62, respectively. Under both the control methods, therefore, the actual energy consumption tended to decrease with increased energy-saving potential or use rate. These results suggest that use of the occupancy-reactive or pre-scheduled control reduces energy consumption of space heating.
Note that there are points whose actual energy consumption were more than 0.60kWh/m 2 in Fig. 9. These points were from the same apartment, whose average used set-point was about 23 • C. Although the apartment actively used the control methods, the actual energy energy use was large because of relatively high set-point. Fig. 10 shows distributions (boxplots) for actual energy consumption per area of space heating under control use. The distributions in Fig. 10 are from three groups: apartments using only manual control, occupancy-reactive control users, and pre-scheduled control users. Apartments with only manual control were those that used neither the occupancy-reactive nor the pre-scheduled control, according to the 1% threshold in Table 2. The sample points for those who used the occupancy-reactive or pre-scheduled control in Fig. 10 were the same as in the data used for Fig. 9. Fig. 10 shows that users of the occupancy-reactive control consumed less energy for space heating. Median values for only manual control, the occupancy-predictive control, and the pre-scheduled control were 0.30kWh/m 2 , 0.20kWh/m 2 , and 0.22kWh/m 2 , respectively. This may be due to the control method itself, but as Fig. 11 shows, the normal-mode set-point temperatures of the occupancy-reactive users were lower than those of the other users. In contrast, Table 3 presents the average difference between the normal-mode set-point temperatures and used set-point temperatures for each group in Fig. 11, which shows that both the occupancy-reactive and pre-scheduled controls reduced set-point temperatures by a lower degree than did only use of manual control. Therefore, the lower energy consumption of the occupancy-reactive control users was probably due not only to the control itself, but to the lower normal-mode set-point temperature specified by the occupancy-reactive control users.
On the condition that the data whose normal-mode set-points were 22.5 • C or higher and the data of Period 4 were excluded, median values of actual energy consumption for only manual control, the occupancy-predictive control, and the pre-scheduled control were 0.29kWh/m 2 , 0.20kWh/m 2 , and 0.14kWh/m 2 , respectively. Here, exclusion of the data whose normal-mode set-points were 22.5 • C or higher is due to avoiding influence by the apartments whose used set-points VOLUME 9, 2021  were high. Exclusion of the data of Period 4 is due to avoiding influence by warm outside temperatures. The average outside temperatures for Periods 1, 2, 3, and 4 were 1.8 • C, 0.8 • C, 5.2 • C, and 8.3 • C, respectively. In addition, the points of the apartments with only manual control in Fig. 11 whose used set-points were much lower than their normal mode set-points were mainly attributed to Period 4. The results described here ascertain more the view that the occupancy-reactive control consumes less energy than only manual control.

D. CONTRIBUTION OF SETBACK TO TOTAL ENERGY-SAVING POTENTIAL
This section describes the extent to which each setback in the occupancy-reactive control contributed to the total energy-saving potential. Here, we compare the additional window-state control described in Section III-C with three TABLE 3. Differences between normal-mode set-point temperatures and used set-point temperatures for each control in Fig. 11.

FIGURE 12.
Boxplots of energy-saving potential of the three occupancy-reactive control setbacks and the additional window-state control. ''Setback-W'' indicates the window-state control described in Section III-C. Number labels on each boxplot indicate the median energy-saving potential of each setback.
setbacks of the occupancy-reactive control. The window-state control was thereby included in (2) for comparison with the other three setbacks of the occupancy-reactive control. Fig. 12 shows distributions (boxplots) of the energy-saving potential of three setbacks of the occupancy-reactive and window-state controls, with Setback W indicating the window-state control. Sample points in that figure were obtained from the apartments in each period with total energy-saving potential of the occupancy-reactive control of 1% or more. Median values for energy-saving potential under each setback, including the window state control, were 10.4%, 21.0%, 0.0%, and 0.4%, respectively. There is an outlier at Setback W that was caused by unusual long-time window opening.
We applied the Wilcoxon rank sum test [43] to confirm statistical differences among the distributions of the energy-saving potential of the setbacks in Fig. 12, which shows the resulting p-values between Setback 1 and each of the other setbacks. The p-value between Setbacks 1 and 2 was 0.62, while p-values between Setbacks 1 and 3 or W were less than 0.01. Therefore, distributions of the energy-saving potential of Setbacks 1 and 2 were not statistically different at the 0.01 significance level, whereas those of Setbacks 1 and 3 or W were statistically different. These results demonstrate that Setbacks 1 and 2 mainly contributed to the total energy-saving potential and that Setback 3 and the windowstate control (Setback W) did not significantly contribute. This is because the former two setbacks were used for much longer time than the latter two. The median use rates of the four setbacks were respectively 20.0%, 28.4%, 0.0%, and 0.5%. 47212 VOLUME 9, 2021 TABLE 4. Numbers of users of the occupancy-reactive and pre-scheduled controls from questionnaire responses, showing the number of users of only manual control. Two users of the occupancy-reactive control who answered using also manual control were counted as occupancy-reactive user. One user of the pre-scheduled control who answered using also manual control was counted as pre-scheduled control user. Table 4 shows numbers of users of the occupancyreactive and pre-scheduled controls from the questionnaire responses. There were 23 valid responses. Two users of the occupancy-reactive control who answered using also manual control were counted as occupancy-reactive user. One user of the pre-scheduled control who answered using also manual control was counted as pre-scheduled control user. Table 4 shows that only 6 of 24 apartments answered that they used the occupancy-reactive control, for a usage ratio of less than one-third. Almost half the apartments used neither the occupancy-reactive nor the pre-scheduled control (only manual control).

E. RELATIONSHIP BETWEEN RESPONSES TO QUESTIONNAIRE AND FREQUENT USERS OF CONTROL
We checked actual usage of the control methods by the users who answered the question about use of control. We found that two out of six occupancy-reactive control users did not actually use the occupancy-reactive control in the experiment and one out of four pre-scheduled control users did not actually use the pre-scheduled control. Three out of twelve apartments that answered using only manual control actually used the occupancy-reactive control in Period 1 or 4. Table 5 shows ratios of the occupants who answered cold in the winter. The ratios were classified by control users (the occupancy-reactive control, the pre-scheduled control, or only manual control), according to the 1% threshold in Table 2. Those ratios were obtained from the data whose normal-mode set-point was lower than 22.5 • C for excluding the data with high set-points, as shown in Section VI-C. Table 5 shows that the occupancy-reactive and pre-scheduled control users tended to feel cold in comparison with the only manual control users in the region where normal-mode set-point was not so high. These results indicate that the users who used either the occupancy-reactive or pre-scheduled control felt cold probably due to the set-points lowered by the control methods, in addition to their low normal-mode set-points.

VII. DISCUSSION
Section VI-B showed that not many apartments used the occupancy-reactive control. Previous studies of occupancydriven controls (including occupancy-reactive controls) tacitly assumed that such controls were fully used by the occupants, because these studies compelled their use. The same holds for studies of equipment scheduling based on optimization techniques [31], [44]. Our findings suggest TABLE 5. Ratios of the occupants who answered cold in the winter. The ratios were classified by control users (the occupancy-reactive control, the pre-scheduled control, or only manual control), according to the 1% threshold in Table 2. In addition, those ratios were obtained from the data whose normal-mode set-point was lower than 22.5 • C for excluding the data with high set-points, as shown in Section VI-C. Each ratio was denoted by the number of the occupants who answered cold to the total number of valid answers classified by control.
that such assumptions are invalid and should be reconsidered. Simulation studies might preferably introduce a usage ratio of occupancy-driven controls for evaluation of more actual usage, because our findings indicate that the evaluation results by previous simulation studies might be excessive.
As shown in Fig. 11, the normal-mode set-points of the occupancy-reactive control users were lower than those of only manual control users. This tendency is probably attributed to their awareness of energy-saving during use of the occupancy-reactive control. Although the number of the occupancy-reactive control users was 10 or fewer in each period, some apartments probably had an energy-saving awareness and tried to use the occupancy-reactive control, which led to 14 apartments using the occupancy-reactive control at least once during the experiment, as described in Section VI-B.
In questionnaire responses, some apartments answered that the behavior of the occupancy-reactive control was hard to understand and seemed strange. While we tried to ensure that occupants understood how to use the occupancy-reactive control, some number might not sufficiently comprehend the occupancy-reactive control for active use. 14 apartments used only manual control throughout the experiment. In addition, as described in Section VI-E, the mismatch found between answers about use of control and actual use of control probably indicates insufficient understanding of the occupancy-reactive control. Our findings suggest that a new approach is necessary for promotion of understanding and active use of the occupancy-reactive control. As described in [45], the first stage requirement of energy-related behavior change is awareness. For obtaining awareness of effectiveness of the occupancy-reactive control, mandatory use of the occupancy-reactive control within a short time such as one week and displaying energy-saving results to the occupants may be useful. For example, it may be helpful to develop a gamification system [45] that enables occupants to check their results and compare their results with those of others by using a figure such as Fig. 9. Note that mandatory use of occupancy-driven controls should be as short as possible, because occupants want to decide on their own will. In this experiment, the occupants preferred our idea that they could select a control from options on their own will. Opt-out of mandatory use should be also allowed for avoiding dissatisfaction of occupants.
The results described in Section VI-E indicate that the occupancy-reactive control users tended to feel cold probably due to not only their low normal-mode set-points but also the set-points lowered by the occupancy-reactive control. However, this study did not obtain sufficient findings about comfort. Although we measured room temperature, room temperatures were collected by unit of 1 • C, and therefore, we did not grasp the details of room temperature change by the control methods. In addition, we did not ask the occupants how they felt when the set-point lowered by a control method was reset to the normal-mode set-point. Clarifying influence of the occupancy-reactive control on occupant comfort in a field study is an important future work. So is the relationship between occupant comfort and active use of the occupancy-reactive control. Collecting thermal comfort of occupants via smart devices such as a smart phone under the occupancy-reactive control is an effective approach.
Studies of energy-saving HVAC control usually compare energy consumption between cases of with and without controls. Figs. 10 and 11 demonstrate that simple energy use comparisons might not clarify the superiority of the occupancy-reactive control, whereas Fig. 9 and Table 3 show that use of the occupancy-reactive control likely contributed to energy-saving. The superiority of the occupancy-reactive control is ensured by careful data analysis as described in the last part of Section VI-C. If we include the data of Period 4 on the condition that we exclude the data whose normal-mode set-points were 22.5 • C or higher, medians of actual energy consumption for only manual control, the occupancy-reactive control, and the pre-scheduled control were 0.20 kWh/m 2 , 0.20 kWh/m 2 , 0.14 kWh/m 2 , respectively. The results described here do not easily support the superiority of the occupancy-reactive control to only manual control. To derive a certain conclusion especially in a field study, such a careful data analysis as shown in the last part of Section VI-C is necessary.
By introducing individual controls based on room thermostats, control at both the house and room level is possible. As this study showed, it may be possible to implement various kinds of setbacks by individual control. Fig. 12 showed that longer use of setbacks is more effective in terms of energy-saving potential. Setbacks expected to be used over long time are thus important for the design of occupancy-driven control.
Our previous study [36] reported that the energy-saving potential of Setback 1 was larger than that of Setback 2. That result was different from the present result in Fig. 12 and obtained only from the second period. This fact suggests that a long-term experiment is necessary for grasping actual aspects of such setbacks as described in this paper.

VIII. CONCLUSION
This paper presented an experimental evaluation of the occupancy-reactive space heating control under an actual environment where occupants used a HEMS. This evaluation was based on a two-month experiment during winter in Lyon, France. In the experiment, we provided occupants with the occupancy-reactive and pre-scheduled controls via the HEMS. They were also allowed to control space heating manually via a thermostat or the HEMS. The occupants could use the controls according to their own will. Control availability was limited in the two experimental periods, but use of available controls depended on the occupants. Our findings showed that the actual energy consumption of space heating tended to decrease with increased energy-saving potential or use rate under the occupancy-reactive control, indicating energy-saving effects by the occupancy-reactive control. Careful data analysis showed that use of the occupancyreactive control consumed less energy than did only manual control. On the other hand, the results showed ten or fewer occupancy-reactive control users in each experimental period, suggesting that the tacit assumption that advanced controls such as the occupancy-reactive control are fully used in previous studies is invalid and should be reconsidered. Promoting comprehension and active use of an occupancy-reactive control is thus an important task in future work toward accomplishing energy-saving by such controls. Evaluation of occupant comfort collected via smart devices under an occupancy-reactive control is another important research topic in a field study.