Information Freshness-Guaranteed and Energy-Efficient Data Generation Control System in Energy Harvesting Internet of Things

In energy harvesting Internet of Things (IoT) systems, the age of information (AoI) should be maintained at a low level to guarantee the accuracy and reliability of derived decisions. In this paper, we design an information freshness-guaranteed and energy-efficient data generation control system (IFE-DGCS) where an IoT gateway with a directional antenna determines the polling frequency for each sector by selecting a polling sector at periodic decision epochs. When polling data, the IoT gateway transfers the RF energy to IoT devices in the polling sector by means of simultaneous wireless information and power transfer (SWIPT). To minimize the energy outage probability while maintaining the AoI below a certain level, a constrained Markov decision process (CMDP) is formulated and the optimal stochastic policy on the polling sector is obtained by a linear programming (LP). To resolve the curse of the dimensionality problem in CMDP, a greedy IFE-DGCS is developed and its performance is extensively studied. Evaluation results demonstrate that IFE-DGCS with the optimal policy achieves a comparable energy outage probability to the conventional energy-oriented scheme while guaranteeing a sufficiently low AoI.


I. INTRODUCTION
Internet of Things (IoT) is rapidly flourishing across the world and it has enabled ubiquitous connectivity among billions of things, ranging from resource-constrained IoT devices (e.g., sensors) to more powerful devices (e.g., smartphones, tablets, and vehicles). In various IoT systems, IoT devices can interact with their environment and transmit a significant quantity of valuable data to a server. For example, in a temperature and humidity monitoring system of a smart factory, IoT devices periodically sense the temperature and humidity (i.e., interact with their environment) and transmit the sensed values to a monitoring server. In these types of IoT systems, outdated data degrade the accuracy and reliability of the derived decisions, thereby causing vulnerabilities related to the safety and security of the systems. In such The associate editor coordinating the review of this manuscript and approving it for publication was Deyu Zhang. cases, the age of information (AoI) [1], which quantifies the freshness of information, can represent the performance of IoT systems. To maintain the freshness at a high level, IoT devices frequently transmit their data to a server. However, the data transmission process consumes a significant amount of energy, and the IoT devices have limited battery capacity [2]; therefore, frequent data transmission can result in energy depletion. Hence, there is a need to determine an appropriate frequency of the data transmission.
In addition, to prevent energy depletion in IoT devices, the energy harvesting technique [3] can be used. 1 The radio frequency (RF) energy harvesting technique, in particular, has garnered significant attention owing to its flexibility [4], [5]. RF energy harvesting can be performed based on omni-directional and directional energy transmissions.
In omni-directional energy transmissions, simpler implementation and operations can be achieved. However, because omni-directional transmissions have lower antenna gain, the IoT devices may not harvest sufficient energy without excessive power consumption of the IoT gateway. Therefore, in this paper, it is assumed that the IoT gateway adopts directional energy transmissions to provide sufficient energy to IoT devices even when they are located far from the IoT gateway [6]. However, owing to the narrow beam of directional energy transmissions, their direction should be carefully determined.
To summarize, in energy harvesting IoT systems, how to collect data (e.g., the frequency of data transmissions of IoT devices) 2 and how to provide the RF energy (e.g., the RF energy transmission direction of the IoT gateway) are important design factors. Accordingly, numerous works in the literature have investigated each factor [7]- [21]. For example, Ko and Pack [16] determined the optimal direction of RF energy transmission considering the number of IoT devices in sectors. Abd-Elmagid et al. [12] obtained the optimal frequency of data transmissions to minimize the weighted sum of AoIs. However, these two design factors are related to each other. For example, when providing RF energy to the appropriate direction, IoT devices can transmit their data more frequently. In addition, when the IoT devices transmit their data infrequently due to a low data generation rate, they do not consume a significant amount of energy; thus, frequent RF energy provision is not required. Consequently, these two design factors should be determined in conjunction. However, to the best of our knowledge, no previous studies have investigated the joint optimization of the frequency of data transmissions of IoT devices and the RF energy transmission direction of the IoT gateway.
In this paper, we design an information freshnessguaranteed and energy-efficient data generation control system (IFE-DGCS). In IFE-DGCS, the IoT gateway with directional antenna determines the polling frequency for each sector (i.e., the frequency of data transmissions of IoT devices) by selecting a polling sector at periodic decision epochs, and IoT devices in the polling sector report their data. When polling the data, the IoT gateway transfers the RF energy to IoT devices in the polling sector via simultaneous wireless information and power transfer (SWIPT). The IoT gateway maintains information such as the estimated energy level and the data generation frequency of IoT devices. Based on this information, it determines the polling sector to minimize the energy outage probability while maintaining the AoI below a certain level. To achieve the optimal performance of IFE-DGCS, a constrained Markov decision process (CMDP) is formulated and the optimal stochastic policy of the polling sector is obtained by linear programming (LP). To resolve the curse of dimensionality problem in the CMDP, a greedy IFE-DGCS is developed. Evaluation results demonstrate that 2 The frequency of data transmissions means the number of transmissions per unit time.
the greedy IFE-DGCS can achieve comparable performance to IFE-DGCS with the optimal policy. Moreover, it was observed that IFE-DGCS operates adaptively according to the operating environment (e.g., data occurrence rate and energy harvesting probability).
The main contributions of this paper are twofold: 1) we design a polling-based data generation control system called IFE-DGCS while jointly optimizing the frequency of data transmissions of IoT devices and the direction of RF energy transmission of the IoT gateway via CMDP formulation; and 2) we analyze and present extensive evaluation results in various environments that can help provide useful guidelines for the design of energy harvesting IoT systems.
The remainder of this paper is organized as follows. Related works are summarized in Section II. Then, IFE-DGCS is presented in Section III. Next, the CMDP model is formulated in Section IV. The greedy heuristic algorithm is proposed in V. The evaluation results are discussed in Section VI, and followed by the concluding remarks in Section VII.

II. RELATED WORK
In energy harvesting IoT systems, how to collect data (e.g., the frequency of data transmissions) and how to provide RF energy (e.g., the direction of RF energy transmission when using directional antenna) are important design factors. Correspondingly, several studies in the literature have investigated these factors [7]- [21]. These works can be categorized into: 1) how to collect data [7]- [15]; and 2) how to provide RF energy [16]- [21].
Oh and Shin [7] proposed a simplified data transmission scheme wherein the IoT devices transmit a data without conducting the connection setup for the radio resource. Fathy et al. [8] proposed data reduction method where IoT devices only transmit their sensed data that deviate significantly from the predicted data. Jarwan et al. [9] designed a data reduction framework and presented dual prediction and data compression schemes to reduce the number of data transmissions. Stojkoska and Nikolovski [10] developed a new coding scheme to compress temporally correlated data and save energy in IoT systems. Mahapatra et al. [11] introduced an energy-efficient error detection and correction scheme to reduce the transmission delay and packet loss rate in large IoT systems. Abd-Elmagid et al. [12] designed an optimal sampling policy to minimize the weighted sum of AoIs in environments where the IoT devices can receive energy from the central node. Zhou and Saad [13] designed an optimal IoT device scheduling policy for minimizing the average AoI of IoT devices considering their data collection capability. Abd-Elmagid et al. [14] formulated a long-term AoI minimization problem and proposed a feasible AoI optimization sampling strategy for real-time monitoring systems. Chi et al. [15] formulated the optimal time allocation problem as a nonlinear optimization problem and suggested an efficient algorithm to minimize the transmission completion time.
Ko and Pack [16] proposed a directional energy transmission algorithm in which the IoT gateway observes the number of IoT devices in sectors and provides RF energy to the sector where many IoT devices exist. Lu et al. [17] formulated a CMDP for mobile devices to determine whether to request RF energy or transmit a packet. Zhang et al. [18] proposed a concept of mobile energy gateway that can move near IoT devices and transmit RF energy; further, they optimized the operation of the gateway based on an Markov decision process (MDP). Joo and Kang [19] proposed a greedy joint scheduling scheme for the transmission of data and RF energy in multi-channel environments. Ko and Pack [20] proposed an energy transmission algorithm for multiple IoT gateways with directional antenna, in which the direction and initial phase for RF energy transmission in IoT gateways are determined to maximize the efficiency of RF energy transmission. Wu et al. [21] formulated a joint optimization problem on relay selection and power control to maximize the throughput with constrained data and energy storage.  Figure 1 illustrates the system model for energy harvesting IoT systems. In general, energy harvesting IoT systems consist of two major components: 1) IoT gateway and 2) IoT device.

III. INFORMATION FRESHNESS-GUARANTEED AND ENERGY-EFFICIENT DATA GENERATION CONTROL SYSTEM
The IoT gateway selects a polling sector at each decision epoch, and then transmits a polling message to that sector by using the directional antenna. At the same time, via SWIPT, the IoT gateway provides the RF energy to IoT devices in the polling sector. According to the beamwidth and direction of the antenna, the geographical region can be divided into several sectors. Figure 1 illustrates an example of an IoT gateway with a directional antenna at an angle of 45 • and eight sectors.
We assume that there are N IoT devices and they generate data (e.g., sense the temperature, monitor the vibration, and etc.) at specific sectors. Meanwhile, IoT devices have the capability of RF energy harvesting, and thus, they can charge their batteries from the transmission of the RF energy at the IoT gateway. Specifically, for SWIPT, it is assumed that IoT devices have power splitting antennas. 3 Therefore, they can split the received RF signal into two power streams, and these streams are forwarded to an information decoder and energy harvester, respectively [23].
From the power stream fed to the energy harvester, IoT devices can harvest and store the RF energy in their energy storage. It should be noted that because RF energy from the IoT gateway can be attenuated owing to the propagation loss and/or some objects, the IoT devices may not harvest energy. 4 In addition, the IoT gateway provides the RF energy to the IoT devices during a short period (i.e., decision epoch). Therefore, the energy harvesting process can be modeled as a Bernoulli random process where the IoT devices harvest one unit energy with the probability p H [26]. Based on the other power stream to the information decoder, the IoT devices can notice that they are polled to transmit their data.
After decoding the polling message, the IoT devices report their data to the IoT gateway using one unit of the energy stored in their batteries. Note that the IoT devices use time-division multiplexing access (TDMA) to transmit their data to the IoT gateway. 5 The IoT devices generate very small size data (e.g., temperature and humidity) and they use the same modulation scheme to transmit one packet regardless of the distance to the gateway (i.e., no adaptive modulation coding (AMC) is used in IoT devices [27]). Therefore, the difference in the transmission times among IoT devices can be neglected, which means that the energy consumption of transmitting one packet is constant regardless of the distance. In addition, due to the small size of the data, the data can always be included in one packet. Therefore, the variation in energy consumption according to the data size can be neglected. If the IoT devices receiving the polling message do not have sufficient energy in their batteries, they cannot transmit their data to the IoT gateway. To alleviate this problem, the IoT gateway maintains an information table comprising the locations (i.e., sectors), the estimated AoIs (i.e., the estimated elapsed time from the new data generation), and the estimated energy levels of the IoT devices. Because we only consider fixed IoT devices, their locations can be easily maintained. The AoIs can be estimated based on the data generation frequency of IoT devices, λ. These values can be synchronized (i.e., the estimated AoIs become zero) when the IoT gateway receives data from the IoT devices. In contrast, the energy levels of the IoT devices are estimated based on the energy harvesting probability p H . Specifically, the energy levels of the IoT devices in the polling sector can be estimated to decrease by a specific portion to transmit their data and increase by a specific portion with the probability p H . This estimated energy level may differ from the actual energy level. To alleviate this situation, IoT devices piggyback their energy levels to the data payload. Based on the information table, the IoT gateway determines the polling sector to minimize the energy outage probability while maintaining the AoI below a certain level. To optimize the performance of IFE-DGCS, we formulate a CMDP model in the next section.

IV. CONSTRAINED MARKOV DECISION PROCESS (CMDP)
For the optimal decision of the polling sector, a CMDP model is formulated in this section. When partially random results need to be constrained and under the control of the agent, the CMDP model can be exploited for decision making [24]. Important notations for the CMDP model are summarized in Table 1.

A. DECISION EPOCH
The timing diagram for the CMDP model is shown in Figure 2. A sequence T = {1, 2, 3, . . .} represents the time epochs when the agent (i.e., the IoT gateway) makes successive decisions. S t and A t denote the state and action selected at the decision epoch t ∈ T , respectively. τ represents the duration of each decision epoch.

B. STATE SPACE
The state space S can be defined as where T i is the state for an estimated elapsed time from the new data generation of the IoT device i (i.e., freshness of data generated by the IoT device i). Furthermore, E i and L i represent the state for denoting the estimated energy level of the IoT device i and the sector where the IoT device i is located, respectively. T i can be described by where T i = 0 denotes that the IoT device i is estimated not to generate new data after the last transmission. On the other hand, T i = 0 represents the estimated elapsed time from the new data generation. In addition, T max is the maximum time handled in the systems. When E max denotes the battery capacity of the IoT device, E i can be defined as (3) L i is described by where N S is the number of sectors where the IoT device can be located.

C. ACTION SPACE
Based on the current state information, the IoT gateway determines the polling sector. Therefore, the action space A can be defined as Note that A can represent the polling sector. For example, A = 3 indicates that the IoT gateway transmits the polling message and provides RF energy to the third sector.

D. TRANSITION PROBABILITY
Because the state transitions of each IoT device are independent of each other, the transition probability with the selected action A from the current state S to the next state S can be described by where S i and S i denote the next and current states of the IoT device i, respectively. T i and E i are influenced by the selected action A. Specifically, when the selected action A is the same as the location of the IoT device i (i.e., the IoT gateway transmits the polling message and provides the RF energy to the IoT device i), the IoT device i transmits its data and harvests energy, indicating the transition of states (i.e., T i and E i ). In addition, because the IoT device i transmits the generated data by consuming its energy, the transitions of T i and E i are affected by each other. Meanwhile, other states independently change with each other. Therefore, the transition probability from the current state, S i = [T i , E i , L i ], to the next state of the IoT device i, S i = T i , E i , L i , can be represented as eq. (7), as shown at the bottom of the next page.
We assume that the inter-data occurrence time of the IoT device i follows an exponential distribution with mean 1/λ i . Then, the data occurrence probability during the decision epoch can be calculated as λ i τ [25]. When the IoT gateway estimates that the IoT device i has not generated data after the last transmission (i.e., T i = 0) and the IoT gateway does not transmit the polling message to the sector of the IoT device i (i.e., L i = A), T i becomes 1 with probability λ i τ . Therefore, the corresponding transition probability is defined as After the IoT gateway estimates that the IoT device i generates new data after the last transmission (i.e., T i = 0) and the IoT gateway does not transmit the polling message to the sector of the IoT device i (i.e., L i = A), T i increases one by one. Therefore, P T i |T i = 0, L i = A, A is defined as When the IoT gateway transmits the polling message to the sector of the IoT device i having energy (i.e., E i = 0 and L i = A), T i becomes 0 regardless of the current value of T i . On the other hand, when the IoT device i is estimated not to have any energy (i.e., E i = 0), it cannot transmit data despite receiving the polling message. Thus, we have and When the IoT gateway does not transmit the RF energy to the IoT device i (i.e., L i = A), its energy status does not change. Therefore, the corresponding transition probability can be represented by However, when the IoT gateway performs polling for a sector of the IoT device i (i.e., L i = A), it can harvest energy. Specifically, the IoT device i can harvest one unit of energy only when its received power is sufficient. 6 Therefore, the Bernoulli random process with the harvesting probability p H ,i of the IoT device i can be exploited to model whether the IoT device harvests energy or not [26]. The IoT device i can harvest one unit of energy when its battery is not full (i.e., E i = E max ). In addition, if the IoT device i has newly generated data (i.e., T i = 0), it transmits the data to the IoT gateway by consuming one unit of energy. Therefore, only when the IoT device i has energy (i.e., E i = 0), it can transmit its data. On the other hand, when there is no newly generated data (i.e., T i = 0), the IoT device i does not consume its energy. To sum up, the corresponding transition probabilities can be denoted by eqs. (13), (14), (15), and (16), as shown at the bottom of the page.
Since IoT devices are generally fixed at specific locations in IoT systems, the transition probability of L i can be defined as

E. COST AND CONSTRAINT FUNCTIONS
To minimize the average energy outage probability, we define a cost function r (S, A). Energy outage implies a situation where the current energy of the IoT device i is 0. Therefore, 6 The power received at the IoT device i (i.e., ηP TX |h i | 2 κ) can change according to the environment. For example, when some objects obstruct the antenna of the IoT device i, its received power can significantly decrease.
VOLUME 8, 2020 the cost function r (S, A) can be defined as where δ [·] is a delta function to return 1 if the condition (i.e., E i = 0) is true. If the condition is not true, the delta function returns 0. The constraint function c(S, A) for the average elapsed time from the last data transmission of the IoT devices (i.e., average AoI) can be represented by

F. OPTIMIZATION FORMULATION
The average energy outage probability and the average elapsed time from the last data transmission of the IoT devices (i.e., average AoI), denoted by ζ E and ζ A , respectively, can be defined as and The CMDP model can be defined as follows: where π is the policy representing the probabilities of taking a particular action at a certain state. In addition, θ A is a desired level of the AoI. The CMDP model can be transformed into an equivalent LP model. When φ(S, A) represents the stationary probability of state S and action A, the equivalent LP model can be expressed as subject to the following constraints: and The objective function in (24) is to minimize the average energy outage probability. The constraint in (25) is to maintain the average AoI below θ A . The constraint in (26) is for the Chapman-Kolmogorov equation. The probability properties can be satisfied by the constraints in (27) and (28).
In this paper, we adopt a stochastic policy for the solution of the CMDP model. Accordingly, action A to be taken at state S is selected randomly according to the optimal probability distribution. Because the optimal solution of the LP model represents the stationary probability φ * (S, A) of state S and action A that achieve the objective of the formulated CMDP model, the optimal stochastic policy π * (S, A) can be obtained from Note that, if A φ * S, A = 0 (i.e., there is no solution to satisfy all constraints), the IoT gateway transmits the polling message to a randomly selected sector. Transmit the polling message and provide RF energy to sector A * 10: end if

V. GREEDY IFE-DGCS
To alleviate the curse of dimensionality problem in the CMDP, we propose a greedy IFE-DGCS as shown in Algorithm 1. First, the IoT gateway counts the number N T A of IoT devices whose estimated elapsed time is lower than the desired level θ A for each sector (line 1). If there is any sector with N T A = 0 (line 2), the IoT gateway selects the preferred sector A * with the largest number of IoT devices whose estimated elapsed time is lower than the desired level θ A (line 3). Then, it transmits the polling message and provides the RF energy to sector A * (line 4). On the other hand, if there is no IoT device with a larger estimated elapsed time than a certain level (i.e., T i < θ A for all IoT devices), the IoT gateway checks the estimated energy levels of the IoT devices (line 6) and counts the number of IoT devices whose energy levels are lower than a desired level of energy θ E (line 7). Based on these numbers, the IoT gateway selects the preferred sector A * where the largest number of IoT devices with    Figure 3 shows an operational example of the greedy IFE-DGCS algorithm, where θ A and θ E are set to 2 and 3, respectively. Note that Figures 3(a) and (b) show the situations at t = a whereas Figures 3(c) and (d) represent the cases at the next time slot, i.e., t = a + 1. First, the IoT gateway counts the number N T A of IoT devices with T i > θ A in each sector. The estimated AoIs of IoT devices 3 and 4 in sector 3 are larger than θ A (i.e., 2). That is, N T A for sector 3 is 2, and it is the largest value among all sectors (see Figure 3(a)). Therefore, the IoT gateway polls the IoT devices in sector 3 and transfers the RF energy to them (see Figure 3(b)). At the next time slot t = a+1, the AoIs of all IoT devices in sector 3 become zero (see Figure 3(c)). Meanwhile, because there is no sector with IoT devices whose estimated AoI is higher than θ A in this time slot, the IoT gateway counts N E A of all sectors (see Figure 3(c)). After that, it polls section 1 with the largest N E A (see Figure 3(d)). Since Algorithm 1 can operate with state information of IoT devices (i.e., S) as shown in the operational example, the space complexity of Algorithm 1 is O(S). On the other hand, to obtain the optimal solution (i.e., to solve the equivalent LP model, we need to store the transition probability whose space complexity is O (S 2 A).

VI. EVALUATION RESULTS
To evaluate the performance of IFE-DGCS, we developed an event-driven simulator using MATLAB and conducted extensive simulations. We compare IFE-DGCS with the following four schemes: 1) RAND where the IoT gateway randomly selects a sector for polling and RF energy transfer; 2) NUM-BER where the polling sector is determined proportionally to the number of IoT devices in each sector; 3) FRESH where the IoT gateway selects a sector where the number of IoT devices with the estimated elapsed time higher than θ A is the largest; and 4) ENERGY where the IoT gateway selects a sector where the number of IoT devices with energy levels lower than θ E is the largest.
Because the objective of this paper is to minimize the average energy outage probability ζ E while maintaining the average AoI ζ A below a certain level, ζ E and ζ A are used as the performance measures of IFE-DGCS. The default parameter settings are summarized in Table 2, where [a, b] denotes a random value between a and b.

A. GREEDY IFE-DGCS VS. OPTIMAL IFE-DGCS
To find out the optimal IFE-DGCS, we employed the brute-force search. Due to the high complexity of the brute-force search, the optimal solution of IFE-DGCS can be achieved only in a small-scale scenario. 7 The desired level of AoI, θ A , is set to 2.  Table 3 presents the effect of the desired level of energy θ E on the average energy outage probability ζ E and the average AoI ζ E . It can be found that both greedy and optimal IFE-DGCS achieve the average AoI below a desired level of AoI (i.e., 2) regardless of θ E . In addition, it can be observed that the greedy IFE-DGCS can achieve comparable performance (i.e., ζ E and ζ A ) to the optimal IFE-DGCS when θ E is set to the appropriate value. Specifically, when θ E is 3, the performance difference between the greedy IFE-DGCS and optimal IFE-DGCS is below 7%. Note that the optimal IFE-DGCS has high complexity due to the brute-force search and thus forthcoming sub-sections consider only the greedy IFE-DGCS for performance evaluations.  Figure 4(a), it is evident that the average outage probability of all schemes decreases with the increase in the energy harvesting probability. This is because a high energy harvesting probability implies that IoT devices can sufficiently charge their batteries despite infrequent transmission of the RF energy. Meanwhile, the average outage probability of IFE-DGCS is maintained at a low level compared to other schemes except for ENERGY. This is because IFE-DGCS provides the RF energy to the sector where the number of IoT devices with lesser energy than the pre-defined threshold, θ E , is the largest. This can help prevent the depletion of energy from IoT devices because most of the IoT devices can have sufficient energy levels that are typically larger than the pre-defined threshold. Note that, because ENERGY provides the RF energy in the same way as IFE-DGCS, its average energy outage probability is slightly lower than that of IFE-DGCS when p H is 0.1 ∼ 0.2. 8 However, ENERGY does not consider the AoI; thus, its average AoI is always higher than that of IFE-DGCS (see Figure 4(b)).
Meanwhile, from Figure 4(b), it can be shown that the average AoI of all schemes decreases as the energy harvesting probability increases. This is because a higher energy harvesting probability implies a smaller probability that the IoT devices cannot transmit their data due to the energy depletion.  observed that IFE-DGCS can minimize the average energy outage probability while maintaining the average AoI below a desired level. This can be explained as follows. In IFE-DGCS, if there is any sector where the IoT devices are estimated to have generated data a long time ago (i.e., the IoT devices have a larger average estimated elapsed time than the desired level, θ A ), the IoT gateway attempts to reduce the average AoI. Specifically, the IoT gateway transmits the polling message to the sector with the largest number of IoT devices whose elapsed time is higher than the desired level (see lines 2-4 in Algorithm 1). Based on this operation, the average AoI does not grow much larger than the desired level. However, if there is no sector where IoT devices are estimated to have generated data a long time ago, the IoT gateway attempts to minimize the average energy outage probability (see lines 5-9 in Algorithm 1).
Because RANDOM and NUMBER do not consider the desired level of AoI, their average energy outage probability and average AoI do not change according to θ A . In contrast, even though FRESH selects a sector considering the desired level of θ A , its performance cannot be comparable to that of IFE-DGCS. This is because FRESH does not consider the energy level of IoT devices and thus, there is a high probability of polling energy-depleted IoT devices.

D. EFFECT OF AVERAGE DATA GENERATION RATE λ
The effect of the average data generation rate λ on the average energy outage probability is shown in Figure 6. From Figure 6, it can be found that the average energy outage probability of comparison schemes except ENERGY increases with the increase in the average of λ. This is because a high data generation rate implies that the IoT devices consume energy whenever they receive the polling message. 9 In this situation, their energy can be easily depleted. On the other hand, because IFE-DGCS and ENERGY transfer the RF energy to the sector that has several IoT devices with low 9 Note that, when no data are generated in IoT devices, they do not need to transmit their energy even though receiving the polling message. estimated energy levels, their average energy outage probabilities are maintained at a low level irrespective of the average data generation rate.

VII. CONCLUSION
In this paper, we propose an information freshness-guaranteed and energy-efficient data generation control system (IFE-DGCS). In IFE-DGCS, the IoT gateway with the directional antenna determines the frequency of data transmission by selecting the polling sector at periodic decision epochs. By means of simultaneous wireless information and power transfer (SWIPT), the IoT gateway can provide RF energy to the IoT device with polling. To minimize the energy outage probability while maintaining the age of information (AoI) below a certain level, we formulate a constrained Markov decision process (CMDP) and obtain the optimal policy for the polling sector via linear programming (LP). To resolve the curse of the dimensionality problem in CMDP, a greedy IFE-DGCS is developed. The evaluation results demonstrate that the greedy IFE-DGCS can achieve comparable performance to IFE-DGCS with the optimal policy. In addition, it is found that IFE-DGCS operates adaptively according to the operating environment (e.g., data occurrence rate and energy harvesting probability). In our future works, we will introduce a deep reinforcement learning approach to operate the proposed system without any information regarding the environments.