Learning Infused Quantum-Classical Distributed Optimization Technique for Power Generation Scheduling

The advent of quantum computing can potentially revolutionize how complex problems are solved. This article proposes a two-loop quantum-classical solution algorithm for generation scheduling by infusing quantum computing, machine learning, and distributed optimization. The aim is to facilitate employing noisy near-term quantum machines with a limited number of qubits to solve practical power system optimization problems, such as generation scheduling. The outer loop is a three-block quantum alternating direction method of multipliers (QADMM) algorithm that decomposes the generation scheduling problem into three subproblems, including one quadratically unconstrained binary optimization (QUBO) and two non-QUBOs. The inner loop is a trainable quantum approximate optimization algorithm (T-QAOA) for solving QUBO on a quantum computer. The proposed T-QAOA translates interactions of quantum-classical machines as sequential information and uses a recurrent neural network to estimate variational parameters of the quantum circuit with a proper sampling technique. The T-QAOA determines the QUBO solution in a few quantum-learner iterations instead of hundreds of iterations needed for a quantum-classical solver. The outer three-block alternating direction method of multipliers coordinates QUBO and non-QUBO solutions to obtain the solution to the original problem. The conditions under which the proposed QADMM is guaranteed to converge are discussed. Two mathematical and three generation scheduling cases are studied. Analyses performed on quantum simulators and classical computers show the effectiveness of the proposed algorithm. The advantages of T-QAOA are discussed and numerically compared with QAOA, which uses a stochastic-gradient-descent-based optimizer.


I. INTRODUCTION
Computational challenges have always played a significant role in the design and operation of complex systems.Power system modernization poses complexity and computational challenges that classical computers and solvers may not meet [1].Generation scheduling is a fundamental problem in power systems, which determines generating units' status to provide the load while satisfying operational constraints cost effectively [2].This problem is mathematically set as a mixed-integer linear programming (MILP) problem [3].As the number of generating units and the penetration of renewables and distributed resources increase, generation scheduling complexity and computational burden increase exponentially.Computationally efficient and scalable approaches must be developed to deal with this problem.
Quantum computing, a rapidly emerging technology, potentially opens new opportunities in solving computationally challenging problems that classical resources may not be able to address [4].Quantum computing uses quantum mechanical laws to perform computation operations.Despite significant progress in recent years, universal error-corrected quantum computers have yet to be achieved.Currently, noisy intermediate-scale quantum technology is available to implement quantum algorithms and recognize problems for which quantum computing outperforms classical counterparts.
Solving combinatorial optimization is a research scope that quantum speedup is expected.This is achieved using methods, such as quantum approximate optimization algorithm (QAOA) [5], Grover algorithm [6], and variational quantum eigensolver [7].The application of quantum computing extends beyond solving combinatorial problems [8], [9].While quadratic unconstrained binary optimization (QUBO) is a well-known class of combinatorial optimization problems that can benefit from quantum computing, it also offers potential advantages for solving a wide range of other optimization problems, such as the quantum interior point method for linear programming and semidefinite programming [10].A QUBO problem can be transformed into an Ising model using a Hamiltonian that includes the weighted tensor product of Pauli-Z operators [11].Limitations of quantum computing algorithms and noisy intermediate-scale quantum devices have initiated the development of hybrid quantum-classical techniques to cope with large-scale problems.The aforementioned quantum algorithms, e.g., QAOA, are designed based on gate-based quantum computers, which are still in the early stage of development and unable to solve large problems [12].In addition, a special-purpose design of quantum computers to solve QUBOs has been developed recently based on quantum annealing [13].Such hybrid algorithms aim to decompose a problem into one subproblem that can be efficiently handled in a classical computer and another subproblem that can be assigned to a quantum processing unit (QPU).Hybrid quantum-classical algorithms can be found in the literature for both the general-and special-purpose problems.
To solve MILP problems, Chang et al. [14] and Paterakis [15] have presented hybrid techniques based on the Benders technique to decompose the original problem into an MILP master problem and a convex linear programming subproblem.In [14], continuous variables are discretized to convert the master problem into a QUBO.This discretization needs ancillary qubits.In [15], a Benders cut selection scheme manages the size of the master problem.The cut selection strategy is a QUBO problem assigned to a QPU.Although Benders-based algorithms guarantee convergence, they take many iterations.A hybrid technique based on the multiblock alternating direction method of multipliers (ADMM) [16] is presented in [17] for MILP problems.The complicating binary variables are relaxed to vary continuously.The problem is split into a QUBO solvable by QPU and continuous blocks solvable by classical solvers.Two-and three-block versions of this method are adopted in [18] and [19], respectively, to solve unit commitment.Although this approach is useful for adjusting the complexity of binary subproblems by encoding them through entangled states encoded within qubits, it might not converge as the problem size increases.Surrogate Lagrangian relaxation is presented in [20] for a hybrid solution of unit commitment.In addition to these algorithms, the authors in [21], [22], and [23] present similar ideas for solving specific problems using quantum computing.The authors in [21] and [22] discretize continuous variables into h parts to turn them into binary variables.This method is not practical as for a problem with n continuous variables, n(h + 1) ancillary qubits are required.In [23], heuristic methods extend the variational quantum eigensolver for solving MILP problems.The main drawback is the inability to guarantee convergence to the global or local optimum.The proposed approach in [24] integrates a classical heuristic algorithm with a quantum annealer to achieve better quality solutions in a shorter time frame than traditional methods.Morstyn [25] converts the combinatorial optimal power flow problem into a QUBO problem, which is amenable to quantum annealing.In [26], a quantum-teaching-learning-based algorithm for optimal energy management of microgrids is proposed, which outperforms other optimization algorithms in terms of convergence speed and accuracy.We note that previous studies have mainly proposed approaches for exploiting quantum computers.The computational process of quantum algorithms, such as QAOA, has not been sufficiently investigated.
The QAOA is a variational quantum algorithm for solving combinatorial optimization problems.It is a promising candidate for demonstrating quantum advantage in the near future [23].The main concept of QAOA is to alternately repeat the cost Hamiltonian, in which its ground state encodes the problem solution and mixing Hamiltonian.It relies on preparing a parameterized quantum circuit on a quantum device and a classical optimizer to find the best quantum circuit parameters.The QAOA is introduced in [5] to solve combinatorial optimization problems.The model and study are derived from a MaxCut problem.Following that work, Lin and Zhu [27] study QAOA's ability to solve more complex problems.The quality of the solution resulting from the QAOA is affected by the quality of variational parameters prepared in a classical optimizer.Therefore, developing effective variational parameter optimization techniques is crucial to achieving quantum advantages.Various approaches are proposed for optimizing variational parameters, including gradient-based [28] and gradient-free methods [29], [30].Neural networks and learning techniques have also been used to optimize the variational parameters [30], [31].Optimization-based methods take many more iterations to achieve optimal results as compared to learning-based methods [32].Scalability is also a key point that learningbased methods cannot address easily.
In this article, we develop a trainable two-loop quantumclassical optimization algorithm for generation scheduling.Generation scheduling is decomposed into one QUBO and two non-QUBO subproblems.The inner QAOA loop solves QUBO on quantum computers.The iterative interactions between the quantum circuit and classical optimizer are translated as sequential time-series-type information.A scalable deep recurrent neural network (RNN) plays the role of an optimizer mimicking the iterative trace between QPU and a conventional computer to determine the optimal quantum circuit variational parameters.With a proper sampling technique, the chosen learner optimizer provides the proposed trainable quantum approximate optimization algorithm (T-QAOA) with the flexibility to converge within a predetermined number of iterations instead of hundreds to thousands of iterations that may be taken by the conventional QAOA.An outer three-block quantum alternating direction method

Engineering uantum
Transactions on IEEE of multipliers (QADMM) loop is designed to coordinate generation scheduling QUBO and non-QUBO subproblems.The inner loop learner stays unchanged at every outer QADMM iteration.The scalability and convergence conditions of the proposed algorithm are discussed.Numerical results on a real quantum computer and quantum simulator show the effectiveness of the proposed trainable two-loop algorithm.
The rest of this article is organized as follows.Section II presents generation scheduling and a preliminary discussion on quantum computing.The generation scheduling decomposition and the T-QAOA are proposed in Section III.Numerical results are discussed in Section IV.Finally, Section V concludes this article.

II. PRELIMINARIES A. COMPACT GENERATION SCHEDULING FORMULATION
Assume that p i,t denotes continuous variables (e.g., power output) and y i,t denotes discrete variables (e.g., on/off status) of unit i at time t.Let I = {1, 2, . .., N} and T = {1, 2, . ..T } be the set of generating units and time periods, respectively.For brevity of notation, we use (p, y) and (p i , y i ) in the following equations, where (p i , y i ) refers to vectors of unit i containing variables (p i,t , y i,t )∀t ∈ T , and (p, y) refers to vectors containing variables (p i,t , y i,t )∀i ∈ I and ∀t ∈ T .The generation scheduling problem is as follows: where objective function (1a) is equal to F (p) + G(y).b i is the fixed cost coefficient of unit i, and c i represents the standby cost of unit i, including no-load cost, start-up cost, and shutdown cost.

B. QUANTUM COMPUTING
Classical computers encode information in binary bits, which are either 0 or 1 s, and use integrated circuits that contain millions of transistors.Quantum computers also operate data as a series of qubits.Unlike regular bits, qubits can simultaneously be at both the |0 and |1 states with a certain probability.Therefore, one qubit can store 2 bits of information.This quality results in a system's exponential scaling advantage regarding the number of required qubits [33].
In a processor with n qubits, 2 n bits of information can be stored.Qubits contain two main properties called superposition and entanglement.Superposition refers to the quantum system's ability to be in multiple states simultaneously, and the correlation between quantum particles is referred to as entanglement [34], [35].
Quantum computers use quantum gates to execute calculations within quantum circuits, as logical circuits and logic gates do in classical computers.Qubits, after initialization, travel through quantum gates, experiencing a rotation, which basically refreshes the probability of each state as |ψ = U|ψ 0 .A typical fixed quantum circuit is shown in Fig. 1(a), where |ψ 0 is the initial state, U represents the equal unitary operator of the circuit at the given angles, and |ψ is the output state.The measurement occurs at the end of the circuit, as states with a higher probability appear more frequently in the measurement.Quantum circuits designed based on free parameters are known as variational quantum circuits (VQCs) [36], as shown in Fig. 1(b).The output of a VQC, |ψ β = U (β )|ψ 0 , is controlled by its variational parameters β.By optimizing this parameter, the VQC conducts different tasks, e.g., solving QUBO problems [5] and system of linear equations [37].
To perform optimization using quantum computers, we should prepare the Hamiltonian of the Ising model or QUBO corresponding to the objective function and constraints of the considered optimization problem.The ground state of a corresponding Ising Hamiltonian is the optimal solution to a QUBO problem [38].The Ising model or QUBO can be represented by a graph G(V, E ), where V and E refer to the set of vertices and edges, respectively.Fig. 2 shows an example of an undirected graph based on which the Hamiltonian Ising model can be constructed.For a random graph G, the Hamiltonian of the Ising model is as follows [11]: where s k is the spin at vertex k ∈ V , J k j pertains to the interaction between qubits k and j, (k, j) ∈ E, and h k is the external magnetic field at vertex k ∈ V .From the physical point of view, vertices are physical qubits, and edges represent the potential locations of two-qubit gates.The lefthand-side argument of the Hamiltonian typically represents the state of the system, i.e., the spins' configuration.In the simplest Ising model, each spin can take one of two values, −1 (spin down) or +1 (spin up).For a system of N spins, the state of the system can be represented as a vector or array s = (s 1 , s 2 , . .., s N ), where each s i is the state of the ith spin and is either −1 or +1.This vector s would be the left-hand-side argument of the Ising model's Hamiltonian.
A QUBO is an energy function that can be transformed into an Ising Hamiltonian ({±1} n ) with a simple conversion of variables [39].Consider the following QUBO with a binary variable x k , linear cost term t k , and quadratic cost term q k j : To convert QUBO ( 4) into an Ising model, the relation . This transformation is applied to all the binary variables that appear in the QUBO function.
In the case of existing soft constraints Q(x) = 0, a QUBO problem can be retrieved by adding a quadratic penalty term ρ||D(x)|| 2 to the objective function based on the penalty method [41], [42].Inequality constraints, such as W (x) ≤ 0, are converted into equality constraints with the help of nonnegative integer slack variables as W (x) + ζ = 0. ζ takes a value as large as − min x W (x).This slack variable is expressed as ζ = l l=0 2 l I l , where l = log 2 (− min x W (x)) represents the number of required bits.Here, I refers to a set of ancillary bits with a length of l.Eventually, the inequality constraints can be treated like equality constraints.In a nutshell, the total Ising Hamiltonian for an objective function C(x), with equality and inequality constraints D(x) = 0 and W (x) ≤ 0, is as follows: where 1 and 2 are large positive coefficients.H obj represents the Hamiltonian regarding the objective function C(x).
The set of binary bits I, which represents the slack variables,

III. THREE-BLOCK QUANTUM ALTERNATING DIRECTION METHOD OF MULTIPLIERS
A decomposition technique is developed to convert the generation scheduling problem (1) into three blocks.The first block formulation is QUBO, which will be solved by the T-QAOA on a quantum computer.The other two blocks are convex optimization problems that can be efficiently solved using classical solvers.The three subproblems are coordinated using the three-block alternating direction method of multipliers (3B-ADMM).Fig. 3 shows a schematic of the proposed solution algorithm.

A. DECOMPOSITION TECHNIQUE
Generation scheduling (1) includes continuous and binary variables.We apply a reformulation to decompose this problem.Binary variables y are relaxed to vary continuously as 0 ≤ y ≤ 1. Solving problem (1) with these relaxed binary variables, however, does not yield accurate results as the variables y may not be resolved to 0 or 1.Therefore, a set of auxiliary binary variables z and continuous variables r is defined.Three sets of constraints (6e)-(6g) guarantee the binary nature of y.Optimization ( 1) is now reformulated as follows: min p,y,r,z Engineering uantum

9:
Dual-variable update: 10: end for 16: Return ( p, z, r).  6) are split into three sets as p = {p, y}, z, and r.With (6e) relaxed, the problem has a block decomposable structure with respect to each set of variables p, z, and r.This means that for a given set of other variables, a smaller subproblem of (6) arises for each variable set.For instance, if z and r are known, we can formulate an optimization subproblem consisting only of variables p.We dualize (6e) using augmented Lagrangian and relax the soft constraints (6f) and (6g) by adding penalty terms to the objective function as follows: where λ denotes dual variables, and σ , ω, and ρ are penalty parameters.We then decompose (7) and solve subproblems according to the procedure outlined in Algorithm 1.
The iteration index m, penalty factors, decision variables, and stopping criteria are set in the initialization step.The first block, called QUBO block, is an optimization problem over the auxiliary variables z given p and r.This block can be assigned to a quantum computer.The second block is a quadratic unconstrained optimization problem over auxiliary variables r given p and z.This problem is not computationally expensive for classical computers as it is convex and unconstrained.The third block, which is a relaxed version of (1) with relaxed binary variables, is a quadratic problem over variables p given z and r.The third block represents a problem significantly simpler to solve than the original problem (1), primarily because it neither contains binary variables nor nonconvex constraints.If the third-block problem is infeasible, it also implies that the original problem (1) is infeasible, and Algorithm 1 stops.Since the QUBO subproblem is a nonconvex problem, the ADMM is generally heuristic.However, Algorithm 1 is guaranteed to converge to a stationary point under some conditions for a large enough ρ > max{σ, ω}.
The selection of parameters ρ, σ , and ω is generally problem dependent, and they must satisfy the condition ρ > max{σ, ω}.It is crucial to choose these parameters carefully to strike a balance between fulfilling constraints and optimizing the objective function.Each parameter affects the weighting of the connected regularization terms within the Lagrangian function.By assigning larger values to these parameters, the solution will incur greater penalties for larger constraint values, thus assisting in the enforcement of soft constraints.However, using larger parameter values could also make the iterative optimization process more challenging.To select these parameters, we aim to align the weights of regularization terms in the Lagrangian function within a similar range.This balance helps in ensuring that no single term overly dominates the optimization process, which might lead to an undesired solution.

1) CONVERGENCE OF MIXED-INTEGER ADMM
The conditions under which Algorithm 1 is guaranteed to converge to a stationary point of the augmented Lagrangian L ρ (7) are the following [17], [43], [44].

1) (Coercivity):
The objective function is quadratic, and all other generation scheduling variables are bounded.

2) (Feasibility):
Since Im(A 3 ) is the entire space, this condition holds for any Im(A T ). 3) (Lipschitz subminimization paths): It is possible to have a constant M > 0 at iteration m such that This condition holds by setting M equal to 1.Note that the same condition is true for variables z and r with M = 1.4) (Objective regularity): The objective is a lower semicontinuous function: Since F ( p) is a sum of convex functions and an indicator function of a convex set, it is restricted prox regular [17].Also with a constant σ , σ 2 ||r|| 2 2 is Lipschitz differentiable.Therefore, this condition holds.
In addition, this algorithm converges to the global optimum if L ρ is Kurdyka-Łojasiewicz function [45], [46].Function (7) satisfies this condition since it is a semialgebraic function.Therefore, by using Algorithm 1, L ρ , which is a soft-constrained version of the problem (1), will converge to a stationary point for a large enough ρ > (σ, ω).The algorithm assumes the non-QUBO and QUBO subproblems to be solved optimally at any iteration, regardless of whether a classical solver or the QAOA method is employed.In addition, interested readers in the above conditions under which problem (6) converges to a global optimum are referred to [17], [43], [45], and [46] for further details.

2) DISCUSSION ON MULTIBLOCK ADMM
We note that fixing r to zero and skipping the second block update turns out to be a two-block implementation of the ADMM.However, including variable r has two advantages.First, to decompose the linear constraint (6c), a three-block implementation is required, and the second block is an identity matrix whose image represents the entire space.It means that for any fixed y and z, there always exists an r that satisfies (6c), and the feasibility of the problem is guaranteed.Second, constraint (6d) can be handled separately from (6c) and return a convex and Lipschitz differentiable term that can be included in the objective function as σ 2 ||r|| 2 2 .Numerical evidence presented by Gambella and Simonetto [17] shows that, in some cases, a two-block implementation of ADMM may converge more quickly than its three-block counterpart.However, the two-block ADMM is prone to nonconvergence and adheres to the local optimality in large problems.

B. DISTRIBUTED COORDINATION
A power system comprises several subsystems with their local computing processors.Power system problems need solution algorithms that can deal with the growing complexity of the system, protect entities' privacy, and provide a solution in a reasonable period of time [47].Meanwhile, near-term quantum computers cannot centrally handle large problems due to their limited qubits.This section addresses a distributed QADMM strategy adaptive to the practical implementation of generation scheduling.
Consider a system equipped with a two-way communication infrastructure enabling data exchange between a system coordinator and i subsystems, as shown in Fig. 4. A local CPU and a QPU are embedded in each subsystem and can optimize, control, and coordinate the operation cycle of their generation units.The system coordinator also has a CPU to coordinate the subsystems.Each subsystem tries to schedule its generation to the system by solving its QUBO and non-QUBO optimizations, while the system coordinator coordinates the generation statuses to meet the system's physical limitations.The distributed coordination process can be outlined as follows.
Step 2: Solve the first, second, and third optimization blocks.QUBO subproblems are solved using a QPU to find z (m) and pass them to CPU for updating r (m) and p(m) by solving second and third optimization blocks.
Step 3: Stop if termination criteria are satisfied; otherwise, go to Step 1.
Generally, generation scheduling has decomposed over units and yields binary and continuous subproblems coordinated through adjusting Lagrangian multipliers iteratively.The continuous subproblems are solved in a classical computer using linear programming methods.The binary subproblems are mapped into Ising models by relaxing constraints, adding their penalty terms to the objective functions, and converting them into QUBO.The derived QUBO subproblems optimize the units' on/off decisions and are solved on QPUs.The continuous subproblems optimize the units' power generation level and are solved on CPUs.

C. QUANTUM APPROXIMATE OPTIMIZATION ALGORITHM
The QAOA finds the solution to the previously developed QUBO block by minimizing the expected value of its Hamiltonian.This expectation function is derived from quantum states that entangle all the possible states in the same probability.The optimal expected value of the Hamiltonian happens by optimizing the rotation angles of quantum gates in a classical solver.
The QAOA acts on n qubits, i.e., 2 n -dimensional Hilbert space, with each qubit representing the state of a binary variable.The quantum process begins with initializing all the qubits at state |0 and then making an equal superposition of all the computational basis states by applying a Hadamard 3102314 VOLUME 4, 2023 Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply. 1 −1 , to each qubit as Given the problem Hamiltonian H c , the QAOA applies a parameterized unitary operator U (H c , γ ), called cost Hamiltonian, depending on angle γ , and then makes a rotation of the resulting state using U (H B , β), called mixing Hamiltonian, depending on angle where these operators are formulated as where variational parameters γ ∈ [0, 2π ] and β ∈ [0, π], or the so-called angles [5], denote the evolution times of the quantum circuit and are utilized to construct a parameterized QAOA circuit.X k refers to the Pauli-X operator, B = N k=1 X k , and p represents the depth of the circuit, i.e., the number of layers for which parameterized U (H c , γ ) and U (H B , β) are repeated.At every layer, the number of cost Hamiltonian depicts edges, and the number of mixing Hamiltonian depicts the number of vertices of graph G. Given the depth of the circuit, 2p angles, i.e., γ = [γ 1 , . .., γ p ] and β = [β 1 , . .., β p ], need to be tuned to make the QAOA yield the optimal result.In practice, a tradeoff determines the number of layers p between the obtained approximation ratio, the parameter optimization complexity, and accumulated errors.Ideally, increasing p enhances the QAOA solution quality, although the complexity of optimizing QAOA parameters with higher p limits its benefits.The QAOA is generally implemented in a quantum circuit as Fig. 5.
After preparing the state |ψ ( γ , β ) , another important component of the QAOA is to calculate the expectation value of H c in the state of |ψ ( γ , β ) .The expectation value is defined as As shown in Fig. 5, variational parameters γ and β are prepared in a classical computer and fed to the quantum circuit iteratively until convergence.Thus, the entire QADMM process involves two loops, as shown in Fig. 3.The outer loop performs the ADMM and incorporates the QAOA, and the inner loop is a hybrid quantum-classical process.A good initialization and optimization process will lead to fewer iterations to find the optimal variational parameters.An overview of the QAOA steps is provided in Algorithm 2. The optimal variational parameter finding strategy introduced in ( 13) and ( 14) is the approach presented by the original QAOA paper [5].The QAOA Algorithm 2 terminates when it fulfills a predefined termination criterion.In a typical QAOA workflow, this criterion is based on the results of the classical optimization loop used to tune the quantum circuit parameters.Setting a maximum iteration, convergence threshold, and time limit is the most common termination criterion for Algorithm 2. After termination, the best solution found during the iterations is returned.Owing to the probabilistic nature of the QAOA and quantum computing in general, the returned solution may not be the absolute optimal solution, but rather a good approximation to it.Typically, different optimizers require hundreds to possibly thousands of quantumclassical iterations to reach a comparable parameter landscape optimum.In addition, the classical optimizer requires more time to obtain the results at every iteration as the size of the expectation function ( 14) increases.These are the major QAOA bottlenecks, which are resolved in the next section.

D. QAOA CIRCUIT DESIGN
Designing a QAOA circuit, as demonstrated in Fig. 5, entails several steps.Central to this process is the construction of the cost Hamiltonian and the mixing Hamiltonian, which significantly dictate the structure of the QAOA circuit.The cost Hamiltonian encodes the problem that we want to solve, and its eigenvalues correspond to the cost function we aim to optimize.For a standard combinatorial optimization problem, after translating the problem into a QUBO or an Ising model, the coefficients and constants from this would form the terms in the cost Hamiltonian.The mixing Hamiltonian enables superpositions of states within the Hilbert space, thereby facilitating a comprehensive exploration of the problem space.
Here are general steps to design a QAOA circuit.Parameters γ and β are optimized using classical computational resources to find the set of parameters that minimize the expectation value of the cost Hamiltonian.This is an iterative process where the parameters are adjusted, the QAOA circuit is run (either on a quantum computer or simulator), the expectation value of the Hamiltonian is calculated, and the process repeats with new parameters.These steps provide a basic guide to designing a QAOA circuit.The specifics of the problem formulation and Hamiltonian definition depend on the problem to solve.

E. TRAINABLE-QAOA
We aim to train a learner to play the role of an optimizer for the QAOA to update the variational parameters.The expected value F ( γ , β ) of problem Hamiltonian is used as the cost function with respect to parameterized state |ψ ( γ , β ) evolved from the QAOA circuit.To choose an optimizer architecture, the QAOA cost function and parameter evaluations are translated over several quantum-classical iterations as a sequential learning problem.RNNs are a type of neural network commonly used to process such sequential information [48].RNNs are networks that take an input vector, create an output vector, and possibly store some information in memory for later use.A particular type of RNN framework that is used for the problem at hand is long short-term memory (LSTM), which has outperformed other RNN architectures in many applications [49].
Fig. 6 shows the structure of the proposed T-QAOA.At an iteration ν, variational parameters (γ , β) ν−1 , the estimated cost function z ν−1 , and the hidden state of the classical network h ν−1 are fed to LSTM from the prior step.LSTM has its trainable hyperparameters φ and employs a generalized mapping as which suggests new variational parameters and a new internal hidden state.After training the weights φ, the new set of generated variational parameters is sent to QPU for evaluation.This loop continues upon convergence.To generate the first query, we arbitrarily fix the variational parameters to a dummy value (γ , β) 0 = (0, 0), implement the QAOA circuit, and set the cost function to the obtained z 0 .
It is important to select an appropriate loss function during training to measure the LSTM performance on the training dataset.We use another loss function L(φ) as which is called "cumulative regret" and is the summation of loss function history at all the iterations uniformly averaged over the horizon.w are coefficients that weigh the progression of the recurrence loop.We set a higher weight for the last steps as they contain more important information.This way, during the first steps of optimization, LSTM is freer to explore a larger section of parameter space, whereas toward the end, it is restricted to choosing an optimal solution.The LSTM optimizer is trained to run for a fixed number of iterations.However, it is possible to allow it to optimize for more iterations than it was initially trained, but later iterations may have weak performance.

Transactions on IEEE
The merit of the proposed LSTM optimizer over other learning-based approaches is its scalability, which our problem desperately needs.Since the QADMM is sequential, it returns the same QUBO problem at every iteration with only some coefficients updated.It means that the QPU faces a slightly different QUBO problem at every ADMM iteration, such that the QAOA circuit remains the same, but the cost Hamiltonian changes, and the parameters of the gates in the circuit will need to be adjusted accordingly.As such, the optimizer should be scalable so that a single set of training data will cover the entire process.Moreover, every standard optimizer follows a step-by-step path to update the variational parameters iteratively.Applying a learner only to predict the optimal variational parameters might yield a warm starting point for variational parameters.LSTM-based learners can be trained to find the paths every standard optimizer might take to reach the optimal values in a few iterations.Therefore, even if their results are not optimal, since they interact with the quantum circuit through predicting process, they provide more efficient starting points for other optimizers that execute local searches.
Another challenge is preparing a dataset to train a learningbased optimizer for large-scale systems.As established in [50], for an LSTM-based optimizer, the training dataset can be driven from the same system with fewer ranges of qubits.Therefore, for a power system with thousands of units, we can train an LSTM optimizer using the dataset driven from a subset of the given system.In addition, the power system topology is prone to change over time by adding or removing lines.In this case, a trained LSTM optimizer is expected to work based on the previous dataset, as it is not sensitive to a slight change in the grid topology.To generate the training dataset, given H c and the fixed number of required qubits, we sample random values of coefficients J k j and h k using independent Gaussian distributions with zero mean and unit variance.Then, the Ising model circuit is constructed using (10) for the sampled Hamiltonian.

F. DISCUSSION ON T-QAOA SOLUTION QUALITY
Using a learner, such as LSTM, to update the variational parameters in each iteration of the QAOA, instead of solving an optimization problem directly, introduces a different approach to optimizing the parameters.In this case, LSTM would learn to predict the optimal parameters based on the given inputs and the desired objective.The quality of the solution obtained using this approach depends on several factors, including the effectiveness of the LSTM model, the amount and quality of training data, and the complexity of the problem being solved.The LSTM model would need to be trained on a suitable dataset that includes inputs (e.g., problem instances) and corresponding outputs (e.g., optimal parameter values) to learn the relationship between the inputs and desired parameter values.The quality of the solution obtained using the LSTM approach would depend on the accuracy and generalization capabilities of the trained model.It is important to note that this approach may have limitations.The performance of LSTM or any other machine learning model is subject to factors, such as the availability and representativeness of training data, the complexity of the problem, and the suitability of the model architecture.In addition, the LSTM-based approach may not guarantee optimal solutions but rather approximate solutions based on the learned predictions.The solution quality achieved using an LSTM-based approach within the QAOA would require thorough evaluation, including comparisons with other optimization methods, benchmarking against known problem instances, and sensitivity analysis.

IV. SIMULATION
Two illustrative mathematical examples and three generation scheduling problems are used to validate the performance of QADMM and the LSTM optimizer.A study of the variational parameters will also be conducted.In the first example, we use a typical MILP problem to show the convergence and correctness of QADMM.In the second example, we use a QUBO problem to study the effect of variational parameters and validate the LSTM optimizer performance by comparing it with a standard optimization technique based on stochastic gradient descent (SGD).In addition, the performance of QADMM and LSTM optimizer is evaluated on a three-unit 3-h generation scheduling example.The results of QADMM are compared with the classical implementation of 3B-ADMM, and the LSTM optimizer performance is compared to SGD.To test the model's scalability, a 24-h generation scheduling problem for 10-and 100-unit systems is solved, and the proposed LSTM optimizer is used to optimize the variational parameters.A noise-free quantum simulator (statevector) is used in combination with IBM's Qiskit [51], Terra, and IBMQ providers.Neural network training and inference are conducted in Keras and TensorFlow [52].

A. ILLUSTRATIVE MATHEMATICAL EXAMPLES 1) EXAMPLE 1
We examine an example of QADMM's performance under simple setups.Consider the following problem [17]: x 1 + x 2 = 1 (17b) The steps in Algorithm 1 are followed to solve (17).We initialize penalty factors as ρ = 1001, σ = 1000, and ω = 900 [17].According to the QADMM direction, we decompose the problem into a QUBO subproblem solved by a quantum computer and continuous and quadratic unconstrained subproblems solved by a classical computer.Fig. 7 shows the reduction of the constraint's residual, defined as r = ||Iy − Iz + Ir|| in Algorithm 1.The algorithm converges after 19  iterations and yields the optimal solution as x = [1, 0, 0] and y = 2 with an optimality gap of zero.

2) EXAMPLE 2
We examine how the QAOA solves a simple QUBO problem.Then, we apply the trained LSTM optimizer to facilitate the process and evaluate its performance.This is a Max-Cut problem on a triangular graph with two edges weighing five and one weighing 1 (see Fig. 8).The objective function is the sum of weights for edges connecting nods between the two subsets, i.e., C(x) = n k, j=1 J k j x k (1 − x j ) and x ∈ {0, 1}.To solve this problem on a quantum computer, we need to translate it into an Ising Hamiltonian form, H c = (k, j)∈E 1 2 J k j (1 − s k s j ) and s ∈ {±1} by taking the relation x k → (1 + s k )/2.We apply the steps explained in Algorithm 2. We create U (H c , γ ) and U (H B , β) from the Ising Hamiltonian and build a one-layer circuit with two variational parameters as in Fig. 9, which includes the following steps.
1) Hadamard gates are applied to the initial state |q 2 q 1 q 0 = |000 .This step prepares the initial state, i.e., an equal superposition of all the possible states 2) The controlled-phase gate is applied to the state |ψ 0 .
In this gate, a specific phase is introduced to the target qubit when the control qubit(s) are in the |1 state.To illustrate the effect of the controlled-phase gate in the first two qubits that includes two controlled-not gates and one controlled-phase gate, we derive the resulting state |ψ 1 as follows: In (18b), a phase shift of −γ is applied to the two-qubit system.As per the first step of the algorithm, the state |ψ 1 is subjected to a Hadamard gate, and the resulting |ψ 1 state can be derived using a two-qubit circuit as follows: (18c) In a similar way, considering a three-qubit quantum circuit depicted in Fig. 9, the state |ψ 1 is formulated as follows: 3) A layer of R x gates is implemented.Specifically, each qubit undergoes a rotation through before the measurement is taken.This results in a −β rotation being applied to the state |ψ 1 .
The measurement of the final state regarding the set of (γ , β) happens after applying the above steps.To address data passing between classical and quantum processors, we build a custom model of an LSTM network.The LSTM optimizer is trained for five iterations, i.e., the CPU and QPU exchange information five times.We set w = 1 5 (0.1, 0.2, 0.4, 0.6, 0.8), giving higher priority to the latter steps in the loop.One hundred data are randomly generated using Gaussian distribution with zero mean and unit variance given the problem's objective function and the 3102314 VOLUME 4, 2023 Authorized licensed use limited to the terms of the applicable license agreement with IEEE.Restrictions apply.number of required qubits.The trained LSTM optimizer performance is compared to SGD in Figs. 10 and 11.The variational parameters for both the LSTM optimizer and the SGD are initialized to (0, 0).LSTM terminates in fewer iterations than SGD.Fig. 10 illustrates the evolution of the cost function over iterative refinements of the variational parameters as proposed by the respective optimizers.Fig. 11 illustrates the path suggested by LSTM and SGD in the space of the parameters.Given the periodic nature of variational parameters, Fig. 11 shows more than one optimal solution.The LSTM optimizer yields the point (γ , β) = (−0.31,0.62) in five iterations, and the SGD yields (γ , β) = (1.26,0.62) in 60 iterations.γ provided by SGD is π/2 ahead of the one provided by the LSTM optimizer.In the dataset used to train the LSTM, there are samples that (γ , β) converge to four different optimal points.The LSTM optimizer learns the periodic behavior and offers the closest optimal point.
In gradient descent, the algorithm starts at an initial point in the parameter space [in our case, variational parameters are initialized to (0, 0)] and iteratively updates parameters to move in the direction of the steepest descent of the cost function.However, owing to the stochastic nature of quantum computing, it is possible to observe nonmonotonic behavior in the optimization process.This implies that the cost function might initially increase before starting to decrease toward the optimal value as we observe in Fig. 11.There are several reasons why this can happen

B. QADMM GENERATION SCHEDULING
This section verifies the correctness and effectiveness of the proposed QADMM and LSTM optimizer as applied to generation scheduling problems.The QADMM is compared with the classical 3B-ADMM and the LSTM optimizer with the SGD optimization approach.We consider three scenarios for each case study.
1) S1: Algorithm 1 is applied to problem (1), and the QUBO block is solved using a classical solver (Gurobi).2) S2: Algorithm 1 is applied to problem (1), the QUBO block is solved using Algorithm 2, and the QAOA classical optimizer is SGD. 3) S3: Algorithm 1 is applied to problem (1), the QUBO block is solved using Algorithm 2, and the QAOA classical optimizer is LSTM (the proposed algorithm).
Note that Algorithm 1's initializations are the same for all the three scenarios.

1) THREE-UNIT 3-H GENERATION SCHEDULING
The system includes three units supplying a demand of 160, 500, and 400 MW at three consecutive hours.There are three subproblems for the problem decomposed over generating units.Each subproblem contains nine binary variables: units on/off, start-up, and shutdown status at time t (3 × 3 = 9).The units' characteristics are given in Table 1.An LSTM optimizer is trained for eight iterations with 800 observations and tested with 200 observations.The optimal status of units and the cost for each scenario are provided in Table 2.The optimal on/off status and the operation cost obtained are the same for all the scenarios.Fig. 12 portrays the reduction of the constraint residual for all the scenarios at every ADMM iteration.The ADMM convergence performance is similar in all three cases.However, there are instances where both S2 and S3 diverged from S1.This could be related to the suboptimal solution of QUBOs after certain outer iterations, and another could be a variation in the configurations of the classical solvers that may induce such discrepancy.We note that perfect optimization or prediction of variational parameters is not a mandatory condition for the QAOA circuit to reach the optimal QUBO solution.In addition, as problem size increases, sensitivity to optimal variational parameters also grows.In the context of this example, given its relatively small scale with nine binary variables and efficient training of the LSTM, both the LSTM and SGD methods are capable of effectively approximating the optimal variational parameters.The residual approaches to the stopping criterion after 25 iterations.In S2 and S3, at every outer ADMM iteration, a hybrid quantum-classical interaction is conducted to find the optimal QAOA variational parameters.Though starting from the same initial point, the LSTM optimizer is trained to terminate after eight interactions, while the SGD optimizer needs 86 iterations on average to converge.Fig. 13 shows the step-by-step moving toward an optimal point in S2 and S3 in the last ADMM iteration, which takes 65 iterations for SGD to converge.

2) LARGE-SCALE GENERATION SCHEDULING
A study of the performance of QADMM and the LSTM optimizer in large-scale generation scheduling problems is conducted.The number of considered units is 10 and 100, and the time horizon has been extended to 24 h.In every case, we have the same number of subproblems equal to the number of generating units.Each subproblem contains 72 binary variables: units on/off, start-up, and shutdown status at time t (3 × 24 = 72).Two LSTM optimizers are trained for ten iterations with 800 observations and tested with 200  observations.The first LSTM is trained using a 72-qubit quantum circuit and a Gaussian distribution of coefficients for the Ising model.The second learner is trained using quantum circuits with a number of qubits in the range of [40,50] and the same Gaussian distribution of coefficients.Table 3 shows the optimal on/off status of five selected generating units of the 100-unit case.The on/off status obtained by QADMM is the same as those of the classical central solution determined by Gurobi.Fig. 14 represents the reduction of the constraints residual for both the 10-and 100-unit cases.As the system scales, the QADMM maintains its convergence capabilities.Scaling up the system results in more subproblems and slower convergence.The ten-unit system converged

Engineering uantum
Transactions on IEEE TABLE 3. ON/OFF Schedule of Five Selected Units of the 100-Unit Problem after 77 iterations, and the 100-unit system converged after 126 iterations.A comparison between the SGD and LSTM optimizers is given in Fig. 15 for subproblem 1 in the last ADMM iteration.Both the approaches started from the same initial point.LSTM optimizers terminate after ten iterations, while the SGD optimizer continues 84 iterations on average to converge.Furthermore, classical optimization takes more time to solve the problem as the dimension of expectation value (13) increases.A comparison between LSTM1, a welltrained optimizer, and LSTM2, a randomly trained optimizer, is shown in Fig. 15, though, from different paths, both the optimizers obtain the optimal results after ten iterations.

V. CONCLUSION
This article presented a scalable two-loop quantum-classical algorithm to solve the generation scheduling problem within the quantum computing framework.A three-block decomposition broke generation scheduling into a QUBO and two non-QUBOs.A T-QAOA solved the QUBO, and 3B-ADMM coordinated computation operations of quantum computer to solve the QUBO and conventional computer to solve non-QUBOs.
Simulation results on two mathematical and three generation scheduling problems showed that the T-QAOA terminates within a predetermined number of iterations, much fewer than that of the traditional QAOA.For instance, for a MaxCut problem, the T-QAOA converged after five iterations, while the traditional QAOA with SGD converged after 60 iterations.For generation scheduling with 100 generators, the T-QAOA took ten iterations to find the optimal results obtained after 84 iterations of the traditional QAOA.In addition, the 3B-ADMM coordinated conventional and quantum computers' computation operations to achieve optimal results of the original generation scheduling problem.The overall QADMM convergence residual was in the range of 10 −6 , which is promising.

FIGURE 1 .
FIGURE 1. Schematic of the (a) fixed quantum circuit and (b) VQC.

FIGURE 2 .
FIGURE 2. Sample graph of a Hamiltonian Ising model.

FIGURE 3 .
FIGURE 3. Overview of the proposed algorithm.

FIGURE 4 .
FIGURE 4. CPU and QPU information flow in distributed QADMM.

Algorithm 2 : 3 :
QAOA Steps.1: Initialize the quantum state by applying the Hadamard operator.2: Model the cost Hamiltonian using the QUBO function.Model the mixing Hamiltonian.4: Create the circuits.5: Run and measure the final state.6: Update the variational parameters γ and β. 7: Go to step 2. The QAOA minimizes the expectation value by updating the quantum state |ψ ( γ , β ) using a classical computer such that the expected function (13) is minimized to obtain the optimal values of γ and β denoted as γ * and β * γ * , β * = arg min γ , β

1 ) 2 )
Problem formulation: Identify and formulate the combinatorial optimization problem as a QUBO or an Ising model.Cost Hamiltonian U (H c , γ ): The cost Hamiltonian encodes the optimization problem, and it typically comprises a combination of Pauli-Z gates and identity gates.This is because many optimization problems can be expressed as a sum of multiqubit Z terms when formulated as an Ising model or a QUBO problem.Depending on the problem, it might need to use controlled-Z gates or more complex operations, such as multicontrolled gates.To implement a term in the Hamiltonian like s i s j (representing a coupling between spins i and j in the Ising model), it could use a circuit consisting of a Hadamard gate on qubit j, a controlled-Z gate using qubits i and j, and another Hadamard gate on qubit j. 3) Mixing Hamiltonian U (H B , β): The mixing Hamiltonian drives transitions between different states in the Hilbert space.This is typically implemented with Pauli-X gates acting on each qubit to perform a bit-flip operation.For more complex QAOA variations, the mixing Hamiltonian can be composed of other gates, such as Y , or a combination of X and Y gates.4) Prepare the initial state: This state is usually a simple quickly prepared state, such as the uniform superposition of all the computational basis states.This is achieved by applying a Hadamard gate to each qubit initialized to |0 state.5) Apply the QAOA circuit: This involves applying U (H c , γ ) and U (H B , β) alternately for a total of p layers.

1 )
Noisy measurements: Quantum computations are susceptible to various kinds of errors and noise, which might impact the measurement outcomes, leading to fluctuations in the cost function.2) Choice of optimizer: Different optimizers have different behaviors.For instance, some might allow for larger steps in the parameter space, which could initially lead away from the minimum.3) Nonconvex cost function: The cost functions in quantum computing problems, including QAOA, are usually nonconvex, meaning that they can have many local minima.The optimizer might temporarily get stuck in a less optimal minimum before eventually finding the global minimum.4) Initial parameters: The choice of initial parameters can influence the path that the optimization takes.If we start at (0, 0), we might initially move in a direction that increases the cost function before eventually finding the correct path toward the minimum.

FIGURE 13 .FIGURE 14 .
FIGURE 13.Contour plot of the expectation function for the three-unit system.

FIGURE 15 .
FIGURE 15.Contour plot of the expectation function for the 100-unit system.