Two-Stage Adaptive Memetic Algorithm with Surprisingly Popular Mechanism for Energy-Aware Distributed Hybrid Flow Shop Scheduling Problem with Sequence-Dependent Setup Time

This paper considers the impact of setup time in production scheduling and proposes energy-aware distributed hybrid flow shop scheduling problem with sequence-dependent setup time (EADHFSP-ST) that simultaneously optimizes the makespan and the energy consumption. We develop a mixed integer linear programming model to describe this problem and present a two-stage adaptive memetic algorithm (TAMA) with a surprisingly popular mechanism. First, a hybrid initialization strategy is designed based on the two optimization objectives to ensure the convergence and diversity of solutions. Second, multiple population co-evolutionary approaches are proposed for global search to escape from traditional cross-randomization and to balance exploration and exploitation. Third, considering that the memetic algorithm (MA) framework is less efficient due to the randomness in the selection of local search operators, TAMA is proposed to balance the local and global searches. The first stage accumulates more experience for updating the surprisingly popular algorithm (SPA) model to guide the second stage operator selection and ensures population convergence. The second stage gets rid of local optimization and designs an elite archive to ensure population diversity. Fourth, five problem-specific operators are designed, and non-critical path deceleration and right-shift strategies are designed for energy efficiency. Finally, to evaluate the performance of the proposed algorithm, multiple experiments are performed on a benchmark with 45 instances. The experimental results show that the proposed TAMA can solve the problem effectively.

SECTION 1 Introduction

As intelligent manufacturing is developing^[1], the pursuit of high efficiency produces huge economic effects while also bringing many challenges to traditional production. Flow shop, as a class of classical shop scheduling problems, is widely used in real production^[2]. However, in many industries, the layout of the flow shop may result in overloading certain stages affecting productivity. Therefore, in reality, in certain industries such as steel, chemical, textile, and electronics, loads are balanced, and efficiency is improved by adding parallel machines at certain stages^[3]. The hybrid flow shop problem (HFSP) can be seen as a union of the flow shop and the parallel machine and is a mainstream manufacturing method^[4]. It needs to consider not only the ordering of jobs but also the choice of parallel machines, which proved to be an NP-hard problem^[4].

As the economy globalizes, traditional centralized manufacturing is hard to satisfy the current demands^{[5], [6]} and distributed manufacturing is getting more and more attention^[7]. Single manufacturing capacity is insufficient, distributed manufacturing is more flexible and efficient, and it has been applied to many shop problems, including distributed flow scheduling^[8], distributed flexible job scheduling^[9], and distributed welding shop scheduling^[10]. The distributed hybrid flow scheduling problem (DHFSP) is an extension of HFSP and has received increasing attention. In reality, many large manufacturing industries such as steel, ceramics, electronics, chemicals, and textiles can be abstracted into DHFSP. Compared to HFSP, DHFSP is more complex because it requires additional consideration of factory allocation. Moreover, the processing capacity of each shop in realistic manufacturing is generally different, so the study of heterogeneous factories is more meaningful^[11]. On the DHFSP with multiprocessor tasks, Ying and Lin^[12] proposed a self-tuning iterative greedy (IG) with an adaptive cocktail decoding mechanism and expanded the field of research on distributed scheduling problems. Existing studies have been effective in solving the DHFSP, but most of them only consider the relatively ideal workshop state. Actual industries are often more complex and need to consider more realistic constraints, such as setup time^[13], machine damage, finite buffers, etc. For example, in many practical applications such as chemical, printing, pharmaceutical, and automotive production, setup time between jobs needs to be considered, and the size of the setup time depends on the previous process on the same machine^[14]. In practice, sequence-dependent setup time (ST) may typically manifest itself in work-piece transport, lathe/machine cleaning, tool change, etc.^[15] However, most articles do not consider the machine setup time or consider the time for sequence-independent setups directly incorporating it into the processing time. In summary, the sequence-dependent setup time is a critical factor affecting productivity and stability, leading to a significant limitation of the practical application of the existing results to production management, and thus deserves a more profound study. Zhao et al.^[16] proposed a reinforcement learning-driven artificial bee colony algorithm to solve the distributed heterogeneous no-wait permutation flow shop problem with ST. The paper proposed to utilize Q-learning for operator selection. Some of the existing studies have taken setup time into account in the shop problem, but there is no effective treatment strategy designed for it. Compared to the flow shop, ST in the hybrid flow scheduling cannot be predicted in advance^[17]. This unpredictability causes difficulties in characterizing some features of the shop problem such as lower bound and thus restricts the use of such methods. Additionally, ST in the hybrid flow brings more challenges to problem decoding and operator design. Therefore, studying the DHSP with setup time is complex and very meaningful.

With global warming, green manufacturing is receiving more and more attention^{[11], [18]}. Manufacturing is the highest energy-consuming industry, so energy-efficient scheduling has also been a critical component of green shop scheduling^[19]. In this context, energy-aware DHFSP (EADHFSP) should be considered^[20]. EADHFSP is more complex as it requires additional consideration of energy consumption and is a multi-objective optimization problem. Dealing with the EADHFSP, a cooperative memetic algorithm (MA) with a learning mechanism was designed by J. J. Wang and L. Wang^[21]. The algorithm accelerates solution convergence by selecting appropriate operators through a policy agent based on reinforcement learning. Most of the existing studies on energy consumption have achieved good effects in considering only processing energy and standby energy, while setup energy should be taken into account in workshop problems that consider setup time, which is more in accordance with the reality of production^[22]. For example, setting energy consumption occurs when performing the removal of previous operations from a container, such as certain types of iron in a torpedo^[23]. We have collected a few studies on setup energy consumption^{[23], [24]} and found that it is difficult to design effective energy-saving strategies to reduce multiple energy consumption at the same time due to the complexity of the problem itself and the need to consider various forms of energy consumption. Therefore, combining the above considerations and to be closer to production in reality, we propose an energy-aware distributed hybrid flow shop scheduling problem with sequence-dependent setup times (EADHFSP-ST), where the energy consumption mainly consists of operation energy, setup energy, and waiting energy. EADHFSP-ST is an enhanced NP-hard problem by considering the three constraints of hybrid flow, distributed heterogeneous factory, and ST simultaneously. It also considers various energy consumption in addition to the makespan, which makes it difficult for the algorithmic design. It can be used in a variety of production scenarios including semiconductor processing, steel manufacturing, and textiles. And as we know, the current studies do not consider all cases simultaneously. Therefore, it is of great practical importance to design an effective algorithmic framework to solve the EADHFSP-ST problem.

It has been shown that MA is a more effective method for solving the shop scheduling problem, which mainly consists of global search along with local search using multiple domain structure operators to promote population convergence^{[25], [26]}. It enhances the evolutionary algorithm by allowing the population to explore different directions in the target space through local refinement operators. Moreover, it has been used for a variety of flow shop problems ^{[27]–​[29]}. According to MA literature, MA can effectively solve complex optimization problems. The location and object of the local search, how to balance the global and local search^[30], and how to choose the appropriate operator for each individual become another issue. The existing MA framework is not effective in solving multi-objective problems, and a two-stage framework for the multi-objective problem can be effective in approaching the Pareto front and exploring more non-dominated solutions^[31]. The two-stage framework performs well by focusing on various metrics at different stages. In the two-stage framework proposed by Ming et al.^[32], a novel subregion dominance and an improved mating selection based on subregion density mainly aim at convergence in the first stage, and a new level-based Pareto dominance cooperates with the traditional Pareto dominance that mainly promotes diversity in the second stage. The two-stage framework is also widely used in shop scheduling problems^{[30], [33], [34]}. Although it is very effective, no studies have applied it to the EADHFSP-ST. Combining the strengths of the MA framework and the two-stage framework, we propose a two-stage MA. However, operator selection is critical to MA performance. Existing MA frameworks always cycle through global search followed by local search, and random selection^[33], polling selection^[35], and confidence-based selection^{[36], [37]} are used for operator selection. All of these studies inherently rely on confidence levels. Confidence-based operator selection mainly determines the selection probability based on the historical experience of all the strategies, which relies too much on historical experience and ignores some efficient operators with lower selection weights, which can be well solved by the surprisingly popular algorithm (SPA)^[38]. SPA is a group intelligent decision-making model. We can abstract the EADHFSP-ST population into a crowd, and the effectiveness of an operator is regarded as its popularity in the crowd. Truly effective operators may have low selection probabilities, so we accelerate the convergence of local search by selecting efficient operators through the SPA. Unlike Li et al.^[38] proposed algorithm, which focused more on updating the latest knowledge, we focus more on the accumulation of historical experience. Summarizing the above considerations, we designed a two-stage adaptive memetic algorithm (TAMA) with a surprisingly popular mechanism, and it is research-worthy.

To better tackle EADHFSP-ST, this paper proposes TAMA to solve the complex shop problem. The goal of EADHFSP-ST is mainly to minimize the makespan and the total energy consumption (TEC). Our contribution is as follows. (1) A TAMA with a surprisingly popular mechanism is designed to solve EADHFSP-ST. The first stage accumulates experience while converging quickly to obtain a better solution, and the second stage gets rid of the local optimum and efficiently selects operators from the elite file to achieve the optimal solution and promote diversity of solutions; (2) A hybrid initialization method is developed with different angles of optimization to ensure the convergence and diversity of the initial solutions; (3) Multiple population co-evolution approaches are proposed for global search, and different evolutionary means are used to balance the exploration and development according to the characteristics of different solutions; (4) Designing problem-specific operators, and non-critical path deceleration and right-shift strategies are designed for energy saving to further reduce energy.

The other sections of this paper are organized as follows. Section 2 lists the related work. Section 3 presents the special related knowledge. The problem description and model construction are presented in Section 4. Moreover, our approach to TAMA is described in detail in Section 5. The numerical test experiments and discussion of TAMA are shown in Section 6 and finally, we summarize our conclusions in Section 7.

SECTION 2 Related Work

Since the 1960s, many scholars have conducted extensive research on scheduling problems^[3]. However, with the rapid development of industrial manufacturing, traditional scheduling can no longer meet the demand, so we can see that the current research focuses on four types of scheduling problems.

The first type is HFSP. Wang^[39] proposed an improved simulated annealing approach to solving HFSP with the goal of lowering overall flow time. In the paper, Nawaz-Enscore-Ham (NEH) heuristics were proposed to develop new solutions, and a tempering function was proposed to improve the global search capability of the solutions. Another related study was presented by Chen et al.^[40], who proposed an algorithm for minimizing makespan. The paper proposed a heuristic-based decoding rule and devised an adaptive crossover and mutation strategy based on sigmoid functions to improve the convergence of the solution.

The second type is the distributed scheduling problem, which aims to improve productivity. On the DHFSP, Shao et al.^[41] designed a multi-domain IG algorithm with multi-search construction to enhance search capabilities. J. J. Wang and L. Wang^[28] proposed a bi-population cooperative MA to solve DHFSP. The proposed optimization framework includes collaborative initialization, bi-directional collaboration, and local reinforcement.

The third type is the green scheduling problem to achieve energy saving and emission reduction. In dealing with realistic and green DHFSP such as prefabricated systems, Li et al.^[42] designed a knowledge-based multi-objective algorithm. This algorithm integrates a new multi-objective optimization algorithm model for environment selection. Li et al.^[35] proposed a fully active scheduling decoding to reduce energy consumption in the flexible job scheduling problem.

The last type is the consideration of setup time, which should be considered in the actual production of steel, semiconductors, and so on. Regarding the consideration of setup time in the shop problem, Shen et al.^[14] developed a tabu search algorithm with diverse structures and specific neighborhood functions for solving flexible job shop problems with ST and proposed a variety of domain structures based on disjunctive graphs. In dealing with fuzzy two-stage hybrid flow shop problems with ST, Cai et al.^[43] proposed a collaborative variable search and for the first time considered both fuzzy number and ST in a multi-factory problem.

SECTION 3 Special Related Knowledge

3.1 Surprisingly Popular Algorithm

SPA is an approach to sociological decision-making proposed by Prelec et al.^[44] Unlike in the past, when the wisdom of the group was utilized directly through the use of democratic decision-making, the SPA does not directly adopt the most popular opinions but rather focuses on protecting the expertise known to the few. For an illustration of the SPA idea, here is a classic example: is Philadelphia the capital of Pennsylvania?^[44] The decision process for this question was as follows: 65% chose “yes”, while 75% were predicted to choose “yes”; 35% chose “no”, while 25% actually knew the answer. It can be obtained that the surprisingly popular degree (SPD) of “yes” is 65%/75% = 0.867, while the SPD value of “no” is 35%/25% = 1.4. We can see that “yes” is more popular, but “no” is more surprising and is the correct answer.

As can be seen, SPA tends to exploit the unpopular but correct minority of expert knowledge that is often used in selection decisions. Cui et al.^[45] used SPA in combination with the particle swarm optimization (PSO) algorithm for improving the traditional PSO and selecting a better solution. SPA has also been used to solve the distributed flexible job scheduling problem in combination with MA^[38]. SPA is a powerful decision model, and it is worth trying to incorporate it into a two-stage framework to guide the solution of the problem of choosing a normal strategy and thus solving the EADHFSP-ST problem.

3.2 Lower Bound With Setup Time

In the EADHFSP-ST problem, three coupled subproblems are included: factory assignment, job sequencing, and speed assignment. Rational optimization of individual subproblems helps in problem decomposition and overall optimization. A typical lower bound of makespan for HFSPs was proposed by Santos et al.^[46] and further studied by Hidri and Haouari^[47]. Considering the objective that assigning jobs to factories with smaller makespan is more conducive to optimizing the makespan, this paper uses the minimum lower bound of the HFSP makespan instead of the optimal value of the factory's makespan to guide the assignment of factories.

For EADHFSP-ST, each factory can be simplified to an HFSP, so the analysis of the lower bound for EADHFSP-ST can be viewed as an analysis of the lower bound for each factory, i.e., an HFSP lower bound analysis. Different from the classical HFSP lower bound calculation method proposed by Santos et al.^[46], we need to additionally consider the setup time as a factor, so the calculation method needs to be certainly improved. We define a head $L_{m}^{j}$ to refer to the total left-hand side processing time of job $j$ before stage $m$, a tail $R_{m}^{j}$ to refer to the total right-hand side processing time of job $j$ after stage $m$, and $\text{ST}_{s,j}^{f}$ to represent the minimum setup time that may be incurred by the s-th stage two-two combinations of job $j$ and other jobs (including the case of the first job in the machine with the virtual job $J_{0}$ ) in the $f$ factory. $P_{s}^{j}$ represents the standard processing time of job $j$ at stage $s,\ V_{s}^{j}$ represents processing speed, and $P_{s}^{j}/V_{s}^{j}$ represents the real processing time. Calculations are referenced in Eqs. (1) and (2). Let $L_{m}^{i}$ and $R_{m}^{i}$ represent the $i-\text{th}$ minimum values of $L_{m}^{j}$ and $R_{m}^{j}$, respectively, $M_{m}$ represent the number of stage $m$ parallel machines, and $J_{N}$ represent the set of jobs in the $f$ factory. The lower bound $\text{LB}_{0}$ for each stage of each factory can be calculated from Eq. (3).

$$\begin{align*} & L_{m}^{j}= \begin{cases} \sum\limits_{s=1}^{m}(P_{s}^{j} / V_{s}^{j}+\text{ST}_{s, j}^{f}), & m > 1; \\ 0, & m=1\end{cases}\tag{1}\\ & R_{m}^{j}= \begin{cases} \sum\limits_{s=m+1}^{N}(P_{s}^{j} / V_{s}^{j}+\text{ST}_{s, j}^{f}), & m < N,\\ (N\ \text{is total stages}); &\\ 0, & m=N\end{cases}\tag{2}\\ & \text{LB}_{0}=\max_{1 \leqslant m \leqslant N}\begin{cases}\frac{1}{M_{m}}\left(\sum\limits_{i=1}^{M_{m}} L_{m}^{i}+ \sum\limits_{j \in J_{N}}(P_{m}^{j} / V_{m}^{j}+\right.\\ \left.\text{ST}_{m, j}^{f})+ \sum\limits_{i=1}^{M_{m}} R_{m}^{i}\right), M_{m} \leqslant\left\vert J_{N}\right\vert; \\ \frac{1}{\left\vert J_{N}\right\vert }\left(\sum\limits_{i=1}^{\left\vert J_{N}\right\vert } L_{m}^{i}+ \sum\limits_{j \in J_{N}}(P_{m}^{j} / V_{m}^{j}+\right.\\ \left.\text{ST}_{s, j}^{f})+ \sum\limits_{i=1}^{\left\vert J_{N}\right\vert} R_{m}^{i}\right), M_{m}>\left\vert J_{N}\right\vert\end{cases}\tag{3}\end{align*}$$

The above lower bound calculation can be regarded as the lower bound of each factory in EADHFSP-ST, which can be used to guide the job to select the minimum lower bound factory, and the heuristic algorithm for initialization in this paper is also designed according to this idea. As far as we know, the makespan of each plant must be greater than the maximum total processing time (including setup time) of all jobs because of limited machine resources. For the lower bound of each factory of EADHFSP-ST, we have another way of defining it as in Eq. (4).

$$\begin{equation*}\text{LB}_{1}=\max\limits_{j \in J_{N}}(\sum\limits_{s=1}^{N}(P_{s}^{j} / V_{s}^{j}+\text{ST}_{s, j}^{f})) \tag{4}\end{equation*}$$

Therefore, we can define the lower bound for each factory of EADHFSP-ST more precisely as $\text{LB}_{f}= \max(\text{LB}_{0}, \text{LB}_{1})$. As for the lower bound of EADHFSP-ST, we can define it as the maximum value of the lower bound for all factories, i.e., $\text{LB}=\max(\text{LB}_{f})$. The calculation of the lower bound $\text{LB}_{f}$ of the factory can well guide the factory assignment of jobs, while the discussion of the lower bound LB of the whole problem can effectively evaluate the performance of the algorithm, and the study of the lower bound is of great significance to shop scheduling.

SECTION 4 Problem Description and Modeling

4.1 Problem Description

The EADHFSP-ST problem is stated as follows. It is composed of $F$ factories with different processing capabilities. Each factory is a series of $m$ stages of hybrid flow shops. There is a set of $n(j=1,2, \ldots,n)$ jobs and $F(f=1,2, \ldots,F)$ factories, each of which has the same $m(s=1,2, \ldots,m)$ stages, and each stage is assigned $l_{s,f}$ parallel machines of the same processing power. Each job $j$ can be arbitrarily assigned to each factory $f$ and processed through all the same stages in a certain order, and at each stage $s$ any machine $i$ can be selected, where job $j$ can choose different speeds $V$ at each stage $s$ processing. Low speed $V_{i}$ has a unit energy consumption of $E_{i}$, high speed $V_{h}$ has a unit energy consumption of $E_{h}$, and the process energy consumption is calculated as $P_{s,j}/V\times E$, where $P_{s,j}$ represents the standard processing time of job $j$ at $s$ stage. And each job $j$ is processed at each stage taking into account the sequence dependence setup time ST and setting the energy consumption ET in addition to the processing time. The problem can be depicted in Fig. 1. Thus EADHFSP-ST consists of four coupled subproblems including factory job assignment, machine allocation of jobs at every stage, sequencing of jobs on every machine, and speed assignment of each job at each stage.

Fig. 1

Gantt chart describing the EADHFSP-ST.

The assumptions of EADHFSP-ST are given as follows:

• All jobs and machines are available at time zero, and the machines are available during processing.

• All machines have the same unit processing power consumption, setting energy consumption, and standby energy consumption.

• The processing capacity of each factory differs mainly in the number of machines at each stage of each factory.

• The number of machines at each stage in the factory is predetermined and can not be changed.

• Each job passes through all the same processing stages, and at each stage any suitable machine is selected.

• Each job has a standard processing time, but the actual processing time is based on its set speed level.

• A higher speed setting usually results in faster processing, proposed by J. J. Wang and L. Wang^[48].

• There is enough buffer in two consecutive stages that it will not be blocked.

• The setup time and setup energy consumption between two jobs are considered, the setup time is related to the sequence of jobs, the energy consumption per unit in the setup is determined.

• No consideration is given to any other break incidents.

4.2 Mixed-Integer Linear Programming (MILP) Model for EADHFSP-ST

We define some signatures of the EADHFSP-ST modeling in Appendix B.

SECTION 5 Our Approach: Two-Stage Adaptive Memetic Algorithm

5.1 Framework of the Algorithm

The proposed TAMA is a Pareto dominance based algorithm with an SPA feedback mechanism, and its framework is shown in Algorithm 1. The description of TAMA is as follows. Firstly, a hybrid initialization strategy is used to generate the initial solution, which enhances the diversity of the initial population while ensuring convergence. It is then divided into two stages, each consuming half the number of evaluations. In the first stage, firstly, the SPA feedback mechanism is used to select the operator for a local search of the whole population. Considering that it is a two-objective problem, environmental selection in the classical non-dominated sorting genetic algorithm-II (NSGA-II)^[49] is used to select better individuals for the next generation and to select the elite archive. Finally, individuals in the archive are decelerated for energy conservation to reduce processing energy consumption. This stage mainly accelerates the convergence of populations and accumulates more experience for the SPA model to be used to guide the second stage. In the second stage, firstly, multiple population co-evolution is used to search the population globally to prevent the population from converging to local optimums^[50]. The environment selection is then performed and the archive is obtained, and the archive uses the SPA probabilistic model selection operator obtained in the first stage to do local search. Finally, the environmental selection is performed again and the archive is obtained, and the archive is decelerated for energy conservation. This stage mainly promotes diversity, so that it can efficiently approach the Pareto and find more non-dominated solutions. Finally, the final population is right-shifted to save energy to reduce standby energy consumption. The first stage in TAMA accelerates population convergence and produces better individual genes, and the second stage promotes diversity, perpetuates and develops good individual genes, and finds more Pareto solutions. Among them, the SPA feedback mechanism makes the operator selection more efficient. Therefore, considering these advantages, we propose TAMA for solving EADHFSP-ST. There are the following innovations in TAMA: (1) A strategy for hybrid heuristic initialization for specific problems is proposed; (2) A crossover strategy for co-evolution is proposed; (3) Problem-specific operators and effective energy-saving strategies are proposed; (4) The SPA-driven two-stage MA framework is applied to the EADHFSP-ST for the first time.

Algorithm 1 TAMA

5.2 Encoding and Decoding

Efficient encoding and decoding approaches can increase the solution space to a certain extent to find hidden efficient solutions while reducing the search space to exclude invalid solutions, and this section focuses on the encoding and decoding methods of EADHFSP-ST. In this paper, we use a representation proposed by Naderi and Ruiz^[51], where $f$ lists are included in the solution and each list represents the order in which jobs are processed in the first stage of the factory. Based on this, the solution is encoded as $[\pi,V_{j,s}]=\{\pi_{1},\pi_{2},\ldots,\pi_{f};V_{j,s}\}$, where $\pi_{f}$ represents the order of operations in the first stage, $f=1,2, \ldots,F. V_{j,s}$ represents the speed setting of job $j$ in stage $s$. For example, for two factories, there are four jobs and a two-stage example. A valid solution can be expressed as $(\Pi,V)$, where $\Pi=\{\pi_{1}, \pi_{2}\}=\{1,4;2,3\}$ and the speed $V$ is set as Eq. (5).

$$\begin{equation*} V=\begin{bmatrix} 1 & 1.5 & 1.5 & 1\\ 1.5 & 1 & 1 & 1.5\end{bmatrix}\tag{5}\end{equation*}$$

The decoding scheme uses the commonly utilized First In First Out (FIFO) rule for job scheduling and the extended first available machine (FAM) rule for machine allocation^[52]. The FIFO rule refers to the first stage of the job being scheduled by the order of coding, the job is first assigned to the specified factory $f$, the processing speed is assigned the speed according to the speed $V$ coding, and the subsequent stages are assigned the processing rule by the priority of the job that is processed first. Extended FAM rules are used for machine assignment, where jobs are assigned to the first available machine taking into account the setup time, i.e., to minimize the setup time to achieve energy savings and to reduce the makespan, and the job is assigned to the machine that has the smallest sum of the current machine's processing completion time and the setup time of the job assigned to that machine.

5.3 Hybrid Initialization Strategy

Heuristic initialization strategies can greatly facilitate the convergence of evolutionary algorithms, so it is very meaningful to design effective initialization strategies for TAMA. To obtain diversity and high-quality initial solutions, a hybrid algorithm of a two-stage heuristic (TSH) and setup time based heuristic (SBH) is designed for different objectives. Based on the high-speed bias for the minimum makespan and low-speed bias for the minimum energy consumption rule, two out of all the solutions generated at the end of the process are selected, and the speeds are set to both high and low speeds to generate more Pareto solutions.

TSH mainly considers the optimization of the makespan and consists of two phases that solve the two subproblems EADHFSP-ST, i.e., factory assignment and job sequencing within every factory. The TSH Phase 1 major factory allocations are designed through the lower bound of the EADHFSP-ST. It mainly consists of two processes, insertion and destructive reconstruction, and the process is as follows:

Insert: A sequence of jobs is randomly generated and sequentially inserted into the factory of the calculated minimum lower bound. To avoid the job always being inserted into a factory and falling into the local optimum, set the tag to record the number of times that the job is inserted into the factory, and when the tag value reaches the number of factories, then consider inserting the job into other factories that can reduce the lower bound of the factory.

Destruction and reconstruction: To avoid convergence of the solution to a local optimum, a destruction and reconstruction process is set up. The destruction process consists of randomly removing $0.05\times n\times F$ jobs from the solution, and the reconstruction process consists of randomly inserting the removed jobs into the solution in order, and replaces the old solution if the resulting new solution has a smaller lower bound on the maximum factory.

The second stage of TSH mainly solves the problem of inner factory job allocation, where the NEH extended NN+MNEH is utilized^[53]. Firstly, consider the scheduling and combination of the first and second jobs in each factory, enumerate all the cases where all the jobs in the current factory are combined two by two, and find the block JF with the minimum makespan under considering only two jobs. The remaining jobs are then sorted in descending order of total actual processing time, and finally, job $j_{k}$ is sequentially inserted into all possible positions of the current factory's identified sequence, choosing to insert into the position with the smallest completion time, and the resulting new sequence is always used for the insertion of the next job.

Unlike the complexity of TSH, SBH is a relatively simple heuristic. SBH mainly considers the optimization of energy consumption yet also considers the maximum completion time. SBH is also divided into two processes: factory selection and intra-factory job sequencing. First, the sequence of jobs is generated randomly, and the jobs are sequentially inserted into the factory with the minimum load. We define TF as the total factory load. TF is calculated according to Eq. (6):

$$\begin{equation*}\text{TF}=\sum\limits_{i, s=1}^{i=n, s=m}((P_{i, s} / V_{i, s}+\text{ST}_{i, s}) / l_{s, f})\tag{6}\end{equation*}$$ where $P_{i,s}/V_{i,s}$ represents the actual processing time of job $i$ at stage $s$. Assuming that each job has the same processing time at every stage ideally, the setup time calculation can be referenced. According to the decoding rule, the previous job of job $j$ on the machine is the one that we randomly select one of the jobs in the sequence before $l_{s,f}$, where $i$ is located to be set as $i$ or if the sequence length is not enough $l_{s,f}$, then $J_{0}$ is set as $i$ and $\text{ST}=\text{ST}_{i,j}$. The position of each inserted factory job is then determined, one at a time, to the position with the smallest total setup time ST, thus determining the sequence of jobs within the factory. The main consideration of SBH is to reduce setup time and factory load to achieve reduced energy consumption and makespan.

In TAMA, half of the solution sequence coding is generated by TSH, and the other half is generated by SBH, where the speeds are randomly coded, and finally, two solutions are randomly selected from the generated solutions to set the speeds of each stage of the operation to high and low speeds, respectively, to generate the initial population. The procedure of hybrid initialization is shown in Algorithm 2.

5.4 Co-Evolution of Multiple Populations

To avoid the randomness of traditional crossover, enrich the search behavior, and prevent falling into local optimums, a global search with different evolutionary approaches for multiple populations is designed, as seen in Algorithm 3. Three populations $P_{i},P_{j}$, and $P_{h}$, superior, intermediate, and inferior, were equally divided based on dominance relationships and crowding, and the evolutionary approach was designed to balance exploration and exploitation based on the different characteristics of the populations.

For $P_{i}$, which contains mainly Pareto solutions, optimization of itself can advance the evolution of the population, which is more biased towards exploitation. Individuals in $P_{i}$ use a greedy insertion domain search operator for self-evolution^[54], while velocity vectors perform random mutation operations. Design the following operations on the critical factory $F_{c}$ with the maximum completion time:

Algorithm 2 Hybrid initialization

Algorithm 3 Cooperative evolution

Greedy insertion: Remove job $J$ at random from $F_{c}$ and insert that job into all possible locations in that factory. If the resulting new solution dominates the old one, the position is inserted to generate a new solution; if all positions do not dominate, the position is randomly inserted to generate a new solution to ensure diversity.

For $P_{j}$, given the proximity of the ministry decomposition to the per-generation Pareto solution, slight perturbations are made in the scheduling and velocity vectors to balance exploitation and exploration. Individuals in $P_{j}$ mainly learn to evolve according to the encoding to each generation of archive Ac (i.e., Pareto solution) using a similar distribution estimation idea.

The $P_{j}$ specific study process is as follows. Firstly, count the set $T=[j_{1},j_{2},\ldots,j_{f}]$ of jobs that appear the most in each factory in Ac, and then count the set $V_{T}=[V_{j_{1}},V_{j_{2}},\ldots, V_{j_{f}}]$ of jobs that appear the most in each stage speed in Ac according to $T$. Then, considering the job insertion factory location and block properties^[16] in $T$, the most neighboring jobs of $T$ are counted, i.e., the set of two adjacent jobs $F$ (previous job) and $L$ (latter job) in the current $T$ in the solution sequence. Finally, the individual in $P_{j}$ first removes the job contained in $T$ and then reinserts the job, and the factory chooses to insert it according to the set $T$. A random choice is made to insert the job in the factory after the job contained in $F$ or before the job contained in $L$ or to insert the position at a randomized location if none of the jobs in the factory belongs to both $F$ and $L$.

For $P_{h}$, this portion of the population is relatively poor and more oriented towards exploration. To ensure diversity, individuals in $P_{h}$ then evolve through a block learning mechanism on the superior population $P_{i}$.

For individual $a$ in $P_{h}$, the specific study process is as follows. Firstly, an individual $b$ is randomly selected from $P_{i}$, and then consecutive blocks Bh of size $L$ (containing the speeds) are taken from the job sequence of its critical factory $F_{c}$. Then, the jobs and corresponding speeds contained in Bh are deleted from $a$, while the remaining jobs and speeds are noted as Jh and Vh, and Bh is copied to the critical factory of $a$. Finally, to ensure better evolution, the jobs contained in Jh in $a$ are taken out in the order of the internal arrangement of the factory and reinserted in order according to the initialization rule SBH, while the remaining velocity vector Vh is randomly generated again. An example of the process description is shown in Fig. 2.

5.5 Local Search Operator

Local search is one of the most critical steps in solving the scheduling problem, which facilitates the optimization ability of evolutionary algorithms^[55], and the domain structure based on critical factory usually has stronger optimization ability. For the EADHFSP-ST problem, this paper designs three common critical factory-based domain structures and two problem-specific block-based search operators^[16]. The population undergoes a localized search to produce individuals that can calculate only the makespan and TEC of the altered factory to improve decoding efficiency. The five operation operators are shown in Fig. 3 and described as follows:

Swap within factory: Randomly select two jobs $J_{1}$ and $J_{2}$ from $F_{c}$ and then swap the positions of $J_{1}$, and $J_{2}$.

Insertion between factory: Randomly remove job $J_{1}$ from $F_{c}$, and then randomly select another factory $f$ and insert $J_{1}$ into $f$.

Fig. 2

Example of block learning evolution.

Fig. 3

Local search operator.

Swap between factory: Randomly select job $J_{1}$ from $F_{c}$, then randomly select another factory $f$ and job $J_{2}$ from it, and then exchange the position of $J_{1},J_{2}$.

Blocks insertion within factory (BI): Randomly remove the job block with block size $L=(0.25N,0.5N) ( >N$ is the number of jobs in $F_{c}$ ) in the critical factory $F_{c}$, and then insert it into the position $p=(1,0.5T)$ before the setup time of all positions in $F_{c} ( >T$ is all insertable positions, and setup time is calculated according to SBH) until the new solution dominates the old one in determining the position, otherwise insert it randomly.

Blocks insertion between factory (BO): Randomly remove the job block with block size $L=(1,0.25T)$ in the critical factory $F_{c}$ and insert it into the factory $F_{\min}$ with minimum completion time of $p=(1,0.5T)$ before the setup time of all the positions until the new solution dominates the old one in determining the position, or else insert it randomly.

5.6 Two-Stage Design with SPA Feedback

Operator selection is a crucial step in the MA framework, and this paper combines the advantages of SPA and two-stage to design a two-stage adaptive operator selection modeling framework with SPD feedback. Prelec et al.^[44] proposed SPA to solve group determination problems. SPA was able to find operators with low selection probability but really effective by correcting the Bayesian probability formula. Therefore, rational allocation of computational resources using the SPA adaptive operator selection mechanism can effectively improve the algorithmic performance.

In the SPA operator selection model, we set two ratios to represent the cognitive gap between the general population and experts. The popularity $(\text{PD}_{i})$ indicates the popularity of the operator, the knowledge of the group, or the historical probability of operator selection. The success probability $(\text{SR}_{i})$ represents the experienced expert cognition that knows the knowledge, i.e., the total effective use probability of the current operator. Most algorithmic operator selection will rely solely on $\text{SR}_{i}$ values for judgment and ignore the effect of $\text{PD}_{i}$ on operator selection. Larger values of $\text{PD}_{i}$ tend to create the false impression that $\text{SR}_{i}$ values are also larger, which in turn ignores the choice of less popular but efficient operators. So we set the $\text{SPD}_{i}$ value to represent the probability of selecting each strategy, and the detailed design process is as follows:

1. Calculation of $\mathbf{SR}_{\boldsymbol{i}}$: We define the number of successful executions of all executions of the operator $i$ as $\text{SN}_{i}$ and the number of failed executions as $\text{FN}_{i}$. Based on all experiences, we can define $\text{SR}_{i}$ as Eq. (7), iter is the number of iterations.

   $$\begin{equation*}\text{SR}_{i}=\frac{\sum\nolimits_{1}^{\text{iter}} \text{SN}_{i}}{\sum\nolimits_{1}^{\text{iter}} \text{SN}_{i}+\sum\nolimits_{1}^{\text{iter}} \text{FN}_{i}}\tag{7}\end{equation*}$$

2. Calculation of $\mathbf{SPD}_{\boldsymbol{i}}$ value: We can define the $\text{SPD}_{i}$ of operator $i$ as Eq. (8).

   $$\begin{equation*}\text{SPD}_{i}= \frac{\text{SR}_{i}}{\text{PD}_{i}} \tag{8}\end{equation*}$$

3. Selection probability update: Similar to the reinforcement learning approach, we update the operator selection probability by defining the reward for feedback. Firstly, the $\text{SPD}_{i}$ value of operator $i$ is calculated. If the $\text{SPD}_{i}$ value is greater than 1, then the reward is given and the operator $i$ selection probability $P_{i}$ is updated to:( $P_{i}$ + reward), otherwise no change is made. The newly obtained probability value $P_{i}$ of the operator is then normalized to obtain the actual selection probability, where $P_{i}$ values less than 0.1 are set to 0.1 to ensure the diversity of the population so that all operators are selected. Finally, the population is randomly divided into five small populations (population size: $\text{ps}\times P_{i}$ ) according to the operator selection probability to execute the operator separately.

The two-stage design takes into account, on one hand, that the SPA model we designed is more dependent on previous experience, and the model selection that has been accumulated through experience will be more accurate. On the other hand, the domain operator can effectively accelerate population convergence, but at the same time, it is easier to fall into a local optimum. This resulted in the design of a two-stage hierarchical framework with an SPA feedback mechanism. The first stage of the population is mainly in the SPA model under the local operator use and stratification under the elite individual deceleration of energy-saving strategy call. The model to accumulate experience while mainly accelerating the convergence, and the local search in the global scope somewhat also ensures the diversity of population evolution. The second stage is somewhat similar to the classic MA framework with elite strategies and with the SPA model adaptive selection operator. In the global search to escape from the local optimum at the same time, the next selected elite individuals local range using the SPA model adaptive selection operator has accumulated experience in the local search mainly to further promote diversity and somewhat accelerate population convergence, and finally, the elite individuals are selected to invoke the decelerated energy saving strategy. The specific process is shown in Fig. 4.

5.7 Energy Saving Strategy

Regarding energy consumption considerations, energy-saving strategies are designed based on two different perspectives, taking into account the impact of the critical path on scheduling. For one thing, speed adjustment, with appropriate deceleration for certain operations that are in the non-critical path, can reduce the energy consumption of the machine in the working state without affecting the makespan. The right-shift operation^[56], and for another, can reduce the energy consumption of a machine in standby mode by right-shifting certain non-critical path jobs on the machine to curtail the waiting time of each machine. The right-shift operation uses the difference between the last job completion time and the first job start time on the main calculator to determine whether energy is saved. So, considering whether the last non-critical path job on the machine moves or not can be judged simply by the presence or absence of a critical path on the machine or by the time difference between the first job move and the last job move on the machine. Figure 5 depicts the operation process of the two methods, and it can be seen that both methods achieve energy savings to some extent.

Fig. 4

Two-stage framework with SPA feedback mechanism.

Fig. 5

Energy saving strategies.

5.8 Time Complexity of TAMA

The proposed TAMA consists of four parts, namely, TSH, co-evolution, local search operator, and energy-saving strategy, and the analysis of its time complexity is as follows:

1. For TSH, the time complexity is $O(1/2\times P\times N^{2}\times M)$, where $P$ refers to the population size, $N$ refers to the number of jobs, and $M$ is the number of machines;

2. For the local search operator, the time complexity is $O(\text{MaxNFEs}\times N\times M)$, and MaxNEFs denotes the number of evaluations;

3. For co-evolution of multiple populations, the time complexity is $O(\text{MaxNFEs}\times N\times M)$;

4. For energy efficient strategy, the time complexity is $O(P\times M\times N)$.

In summary, the complexity of the TAMA algorithm is $O(P\times M\times N^{2})$.

SECTION 6 Experimental Result

To validate the efficiency of the proposed TAMA in resolving the EADHFSP-ST, all experimental designs are coded in MATLAB and performed in the same computer environment using Intel Core i7-11700 CPU @ 2.50 GHz 2.50GHz, 16GB RAM under Windows 10 operating system.

6.1 Experimental Metrics

EADHFSP-ST is a multi-objective optimization problem (MOP). So we choose three MOP metrics, hyper volume (HV)^[57], generation distance (GD), and Spread^[49], to evaluate the algorithm's performance.

HV metric denotes the volume of the area in the target space surrounded by the set of non-dominated solutions obtained by the algorithm and the reference point. The higher the value of HV, the better the overall performance of the algorithm. It is calculated as Eq. (9).

$$\begin{equation*} HV(G, R)=\bigcup\limits_{a \in G}^{G} v(a, R)\tag{9}\end{equation*}$$ where $G$ represents the Pareto frontier (PF) computed by each algorithm and $R$ is the reference point for all solutions obtained by all algorithms. The $R$ value is usually set to (1.1, 1.1) for better HV calculation. $a$ is a non-dominated solution in the normalized PF. $v$ is the volume of the hypercube enclosed by the boundaries $a$ and $R$.

Both GD and Spread metric consider the Euclidean distance between points. The GD metric stands for the average minimum distance representation from each point in the solution set $P$ to the reference set $P^{\ast}$. A lower GD metric indicates better convergence, which is calculated as in Eq. (10). The Spread metric is used to indicate diversity, with a smaller Spread metric indicating better distribution, which is calculated as in Eq. (11).

$$\begin{gather*}\text{GD}(P^{\prime}, P^{\ast})=\frac{\sqrt{\sum\limits_{y \in P^{\prime}} \min\limits_{x \in P^{\ast}} d(x, y)^{2}}}{\vert P^{\prime}\vert }\tag{10}\\ \text{Spread}=\frac{d_{f}+ d_{l}+ \sum\limits_{i=1}^{N-1}\vert d_{i}-\bar{d}\vert }{d_{f}+ d_{l}+(N-1) \bar{d}}\tag{11}\end{gather*}$$ where $P^{\ast}$ denotes the reference PF obtained from all MOEAs, $P^{\prime}$ represents the set of nondominated solutions obtained from each MOEA, $d$ represents the Euclidean distance between neighboring points in the PF, and $d(x,y)$ represents the Euclidean distance between points $x$ and $y$.

6.2 Experimental Instance

The experimental examples are mainly used to evaluate the performance of TAMA. Since there is no benchmark directly applicable to EADHFSP-ST, with some modifications based on the benchmark proposed by J. J. Wang and L. Wang^[21], a suitable benchmark is designed in this paper. The number of factories $F$ is from {2, 3, 4, 5, 6}, the number of jobs $n$ is from {20, 50,100}, and the number of stages $m$ is from {2, 5, 8}. The number of parallel machines $l_{f,s}$ per stage $s$ in each factory $f$ is sampled from the discrete set {1, 2, …, 5} sampled uniformly at random. The standard processing time $P_{j,s}$ for each job $j$ at each stage $s$ is a discrete value sampled uniformly from Zhang and Chiong^[58] and Wu et al.^[59] Based on Ref. [16], the setup time $\text{ST}_{j,s}$ between each job pair in each stage (including the virtual job $J_{0}$ ) is designed to follow a uniform distribution $u$ ^{[1], [50]}. According to the knowledge, wait for the energy rate to be set to 1 and set the energy rate to 1.5. The pairs of speed and processing energy consumption rate relationships are set to {1:5; 1.5:10} based on literature and realistic experience. From this, a total of 45 instances of $(F,n,m)$ are generated from it. In addition, the benchmark can be obtained from https://github.com/PPT842/Instance.

6.3 Parameter Calibration

The efficiency of the algorithm is somewhat affected by the configuration of the parameters, and this section focuses on finding suitable parameter settings for TAMA. To find the optimal parameters, this paper adopts Taguchi's design of experiments (DOE) methodology^[60]. TAMA has three main parameters, including the size of the population ps, the proportion of blocks in co-evolution to the length of critical plant operations BL, and the size of the reward setup of the SPA model $R$. An orthogonal experiment $L9(3^{3})$ was designed based on three-parameter levels. Based on the analysis of the algorithm, the parameter levels are set as follows to ensure the performance of the algorithm: ps = {80, 100, 120}, BL = {(0, 0.25), (0.25, 0.5), (0.5, 0.75)} (0 means the length of the block is 1), and $R=\{0.05,0.1,0.15\}$. To ensure the fairness of the experiment, all parameter combinations were executed 20 times with the same stopping criterion $(\text{MaxNFEs}=2\times 10^{4})$. Figure 6 illustrates the main effect plots for all indicators of the above three parameters. The larger the value of HV, the better the performance of the algorithm. The GD and Spread metrics, on the other hand, have smaller values for smaller algorithm performance. And HV and GD indicators should be considered relatively more in this category. On a comprehensive analysis, the optimal settings of the parameters were ps = 100, BL = (0.25,0.5), and $R=0.1$.

6.4 Effectiveness of Each Improvement Part of TAMA

This section is mainly used to validate the effectiveness of each improved part of the algorithm, and nine variants of the TAMA are designed. The five algorithms to verify the effectiveness of each strategy are designed as follows. The TAMA1 represents the TAMA without an initialization strategy. TAMA2 represents the population in the co-evolution of multiple population in TAMA divided into three sub-populations in a randomized way. The TAMA3 represents the TAMA without multiple population evolutionary mechanisms, but no population delimitation and using the TAMA that designs an evolutionary approach with the classical single-point crossover and polynomial variation ideas based on the coding approach. The TAMA4 represents the TAMA with no energy-saving strategy. And TAMA5 represents TAMA without block localized search strategy. The four algorithms to verify the effectiveness of the two-stage framework with SPA feedback mechanism are designed as follows. The MA represents the traditional MA framework, the TMA represents the two-stage framework without the SPA model, the SPAMA1 represents the framework with the SPA model but only the first stage, and the SPAMA2 represents the framework with the SPA model but only the second stage. For fairness of comparison, all variants were run 20 times with the same stopping criterion $(\text{MaxNFEs}=2\times 10^{4})$ in all instances. The values of the statistical metrics for 20 runs of 45 instances are listed in Tables A1, A3, and A4 in the Appendix, where the symbols “+/−” indicate whether the variant algorithms' superiority or inferiority over the instances is significant. The symbol “=” means that the algorithm is not significant on the instance. The algorithm's optimal mean will be labeled black. The last line counts the number of symbols “−/=/+”. The Friedman rank detection results of the TAMA algorithm and its variant algorithms are recorded in Table 1 with a confidence level of $a=0.05$. Combining the number of “−/=/+”, the average number of labeled blacks, and the rank ranking. It can be seen that TAMA has the highest rankings in the GD and HV metrics and is superior to the other seven variants of the algorithm, while the Spread metric is not important because it has an elite archiving strategy and multiple local search operators to increase diversity. Figure 7 shows a convergence plot on the HV metrics of the variant algorithm on two randomized instances. It can be seen that in contrast to the other variants, the TAMA algorithm can converge quickly, mainly due to the design of the initialization and the adaptive local search with the SPA feedback mechanism, and can get rid of the local optimum to some extent, which is mainly due to the use of global search in the second stage. In sum, the algorithm effectively enhances diversity and convergence.

6.5 Comparison and Discussion Among Algorithms

To further validate the effectiveness of TAMA, it is compared with some existing algorithms. Considering that there is no direct algorithm applied to the EADHFSP-ST problem, this paper compares TAMA with some mainstream multi-objective algorithms (MOEA), including NSGA-II^[49], SPEA-II^[61], MOEA/D^[62], and ARMOEA^[63]. There is also a decoding modification of the recently proposed KMOEA^[42] for solving multi-objective distributed homogeneous hybrid flow shop added to the comparison experiments. These algorithms were chosen considering that they are either classical algorithms for solving hybrid flow scheduling problems or newer and efficient methods for solving bi-objective problems, as well as the latest algorithms for multi-objective distributed hybrid flow shops. The parameters of NSGA-II, SPEA-II, MOEA/D, ARMOEA, and KMOEA are set according to the settings in the corresponding papers. To ensure the fairness of the experiments, all comparison algorithms were run 20 times with the same stopping criterion $(\text{MaxNFEs}=2\times 10^{4})$ in all instances.

Fig. 6

Main effects plot of three metrics: HV, GD, and Spread.

Fig. 7

Comparison of convergence plots of HV metrics of the algorithm with other variants. NFEs is the number of function evaluations.

Table 1 Overall ranks through the Friedman test within all variants (a level of significant $\boldsymbol{\alpha}=\mathbf{0}.\mathbf{05}$ ).

Table 2 shows the results of the Friedman ranking test for all comparative algorithms in all instances, where the confidence level $a=0.05$. From Table 2, we can see that TAMA ranks first and is significant in both GD and HV indicators. The Spread metric is ranked the second, not better than KMOEA, but the difference is not significant and superior to other algorithms. Tables A2, A5, and A6 in the Appendix, list the values of the statistical metrics for 20 runs of 45 instances, where the symbols “+/−” indicate that the algorithm is significantly better or worse at the instance, and the symbol “=” means that the algorithm is not significant at the instance. We can see that TAMA significantly outperforms the other algorithms in more than half of the instances for both the HV and GD metrics. As for the Spread metric, the algorithm's proposed elite strategy and diverse operators can produce more Pareto solutions, which is slightly worse than KMOEA but also better than other algorithms to remain somewhat competitive.

Table 2 Overall ranks through the Friedman test within all comparison algorithms (a level of significant $\boldsymbol{\alpha}=\mathbf{0}.\mathbf{05}$ ).

TAMA's success is in design. First, the two-stage MA framework with the SPA feedback mechanism balances both convergence and diversity. Second, the heuristic hybrid initialization method produces more high-quality initial solutions. Third, multiple swarm co-evolutionary global search approaches are simultaneously considered for development and exploration. Fourth, the critical path-based energy-saving strategy can effectively reduce TEC. Finally, the design of five problem-specific-based operators and elite archives greatly improves diversity while somewhat enhancing convergence.

In addition, Fig. 8 illustrates the Pareto fronts obtained by all the compared algorithms in randomized instances. From the figure, we can see that the two objectives have a clear conflict, and from the PF we can see that TAMA is significantly better than the other algorithms in the aspect of convergence.

SECTION 7 Conclusion

In this paper, we propose a two-stage adaptive MA with the surprisingly popular mechanism for the multi-objective distributed green heterogeneous hybrid flow shop scheduling problem considering setup time and energy consumption. Firstly, a hybrid initialization approach is designed to generate quality initial populations. Second, a global search balanced development and exploration of multiple population co-evolutionary approaches is proposed. Thirdly, the design of a two-stage selection of more efficient operators with SPA feedback simultaneously balances convergence and diversity. Next, a critical path based energy saving strategy is proposed to reduce the TEC. Finally, from the experimental results, it is clear that TAMA shows a good performance in solving the problem.

Fig. 8

PF comparison results on $\boldsymbol{F}_{\mathbf{4}}\_\boldsymbol{n}_{\mathbf{50}}\_\mathbf{s}_{\mathbf{8}}$ instance.

In future work, we can consider the following tasks. (1) Carry Out more research on distributed heterogeneous hybrid flow shop in the context of real production, and consider more constraints including no-waiting, blocking, no-idle, and some dynamic events (machine damage, order insertion, etc.). More objectives can also be considered. (2) Extend the use of the two-stage MA framework with SPA feedback mechanism by applying it to other shop problems. (3) Consider the integration of other learning mechanisms with the evolutionary algorithms, e.g., deep reinforcement learning. (4) Design a more excellent framework to balance the global and local search.

ACKNOWLEDGMENT

This work was partly supported by the National Natural Science Foundation of China (No. 62076225).

Appendix

SECTION A Statistical Result

Statistical results for all algorithm variants and for all comparison algorithms for the means of the GD, HV, and Spread metrics in all instances are shown in Tables A1–​A6.

SECTION B MILP Model for EADHFSP-ST

Here we define some signatures for modeling EADHFSP-ST.

Index:

$n$:	Number of jobs.
$F$:	Number of factories.
$m$:	Number of stages.
$M$:	Number of machines.
$j$:	Index of jobs, $j\in\{1,2,\ldots,n\}$.
$s$:	Index of stage, $s\in\{1,2, \ldots,m\}$.
$f$:	Index of factories, $f\in\{1,2, \ldots,F\}$.
$i$:	Index of machine, $i\in\{1,2, \ldots,M\}$.
$d$:	Index of speed.
$r$:	Index of processing locations on the machine.

Parameter:

$J_{0}$:	Virtual job for counting the first job setup time on the machine.
$l_{s,f}$:	Number of machines in stage $s$ of the $f-\text{th}$ factory.
$p_{j,i}$:	Standard processing time for job $j$ on machine $i$.
$v_{d}$:	Speed level setting.
$n_{d,i}$:	Total number of speeds on machine $i$.
$n_{r,i}$:	Total number of positions on machine $i$.
$M_{s}$:	The set of machines at stage $s$.
$\text{EPU}_{i,v_{d}}$:	Energy consumption rate for unit processing of machine $i$ at rate $v_{d}$.
$\text{ESU}_{i}$:	Standby unit energy consumption rate for machine $i$.
$\text{ETU}_{i,j,q}$:	Setup energy consumption of units between job $j$ and $q$ on machine $i$.
$\text{ST}_{i,j,q}$:	Setup time between jobs $j$ and $q$ on machine $i$.

Decision variable:

$S_{j,s,f}$:	Start time of job $j$ in factory $f$ at stage $s$.
$C_{j,s,f}$:	Completion time of job $j$ in factory $f$ at stage $s$.
$\text{SM}_{i,r,f}$:	Start time of the $r\text{-th}$ processing position on machine $i$ in factory $f$.
$\text{CM}_{i,r,f}$:	Completion time of the $r\text{-th}$ processing position on machine $i$ in factory $f$.
$\text{EP}_{i,f}$:	Total processing energy consumption of machine $i$ in factory $f$.
$\text{ES}_{l,f}$:	Total standby energy consumption of machine $i$ in factory $f$.
$\text{ET}_{i,f}$:	Total setup energy consumption of machine $i$ in factory $f$.
TEC:	Total energy consumption.
$Y_{i,j,s,r,f,v_{d}}$:	A 0/1 variable. The variable is 1 if $j$ is machining at $v_{d}$ speed at the $r-\text{th}$ position on machine $i$ in stage $s$ of factory $f$.
$Z_{j,s,f}$:	A 0/1 variable. Set to 1 if $j$ is assigned to the $s$ stage of the $f$ factory.
$U_{i,j,s,r,f}$:	A 0/1 variable. Set to 1 if $j$ is assigned to the $r-\text{th}$ position of machine $i$ in stage $s$ in factory $f$.
$H$:	$H$ is a very large positive number.

The MILP model of EADHFSP-ST that minimizes the makespan and TEC is expressed as follows:

$$\begin{gather*}\text{minimize}(C_{\max}, \text{TEC})\tag{A1}\\ C_{\max}\geqslant C_{j,s,f},\forall j,f, s\tag{A2}\\ \text{SM}_{i,r,f}\geqslant 0,\forall i\in M_{s}, r,f\tag{A3}\\ \text{CM}_{i,r,f}\geqslant \text{SM}_{i,r,f},\forall i\in M_{s}, r,f\tag{A4}\\ s_{j,s,f}\geqslant 0,\forall j,s,f\tag{A5}\\ C_{j, s, f} \geqslant S_{j, s, f}, \forall j, s, f \tag{A6}\\ \text{CM}_{i, r, f} \leqslant \text{SM}_{i, r+1, f}, \forall i, 1 \leqslant r< n_{r, i}, f \tag{A7}\\ \text{SM}_{i, r+1, f}-\text{CM}_{i, r, f}+H \times(1- Y_{i, j, s, r+1, f, V_{d}}) \geqslant 0 \\ \forall i \in M_{s}, r, j, s, f, d\qquad\qquad\qquad\qquad\qquad\qquad \tag{A8}\\ \text{SM}_{i, r, f}+ \sum\limits_{j=1}^{n} \sum\limits_{d=1}^{n_{d, i}} p_{j, i} / v_{d} \times Y_{i, j, s, r, f, v_{d}}=\text{CM}_{i, r, f}, \\ \forall i \in M_{s}, r, s, f\qquad\qquad\qquad\qquad\qquad\qquad\qquad \tag{A9}\\ C_{j, s, f} \leqslant S_{j, s+1, f}+H \times\left(1- \sum\limits_{r=1}^{n_{r, i}} Y_{i, j, s+1, r, f, v_{d}}\right),\qquad \\ \quad\forall j, 1 \leqslant s, i \in M_{s+1}, f, d\qquad\qquad\qquad\qquad\qquad\qquad\ \tag{A10}\\ \sum\limits_{f=1}^{F} \sum\limits_{i \in M_{s}} \sum\limits_{r=1}^{n_{r, i}} \sum\limits_{d=1}^{n_{d, i}} Y_{i, j, s, r, f, v_{d}}=1, \forall j, s\tag{A11}\\ \sum\limits_{f=1}^{F} Z_{j, s, f}=1, \forall j, s \tag{A12}\\ Z_{j, s, f}= Z_{j, s-1, f}, \forall j, 1< s \leqslant m, f \tag{A13}\\ \sum\limits_{j=1}^{n} \sum\limits_{s=1}^{m} \sum\limits_{d=1}^{n_{d, i}} Y_{i, j, s, r, f, v_{d}} \leqslant 1, \forall i \in M_{s}, r, f \tag{A14}\\ \sum\limits_{j=1}^{n} Y_{i, j, s, r, f, v_{d}} \geqslant \sum\limits_{q=1}^{n} Y_{i, q, s, r+1, f, v_{d}}, \\ \forall i \in M_{s}, 1 \leqslant r< n_{r, i}, s, d\qquad\ \ \tag{A15}\\ \text{SM}_{i, r, f} \leqslant S_{j, s, f}+H \times(1- Y_{i, j, s, r, f, v_{d}}), \\ \forall i \in M_{s}, r, j, s, f, d\qquad\qquad\qquad\qquad\tag{A16}\\ \text{SM}_{i, r, f}+H \times(1- Y_{i, j, s, r, f, v_{d}}) \geqslant S_{j, s, f}, \\ \forall i \in M_{s}, r, j, s, f, d\qquad\qquad\qquad\qquad\tag{A17}\\ \text{CM}_{i, r, f} \leqslant C_{j, s, f}+H \times(1- Y_{i, j, s, r, f, v_{d}}), \\ \forall i \in M_{s}, r, j, s, f, d\qquad\qquad\qquad\qquad \tag{A18}\\ \text{CM}_{i, r, f}+H \times(1- Y_{i, j, s, r, f, v_{d}}) \geqslant C_{j, s, f}, \\ \forall i \in M_{s}, r, j, s, f, d\qquad\qquad\qquad\qquad \tag{A19}\\ \sum\limits_{i \in M_{s}} \sum\limits_{r=1}^{n_{r, i}} \sum\limits_{d=1}^{n_{d i}} Y_{i, j, s, r, f, v_{d}}= Z_{j, s, f} \leqslant 1, \forall j, s, f \tag{A20}\\ \quad\text{EP}_{i, f}= \sum\limits_{j=1}^{n} \sum\limits_{s=1}^{m} \sum\limits_{r=1}^{n_{r, j}} \sum\limits_{d=1}^{n_{d, i}} p_{j, i} / v_{d} \times Y_{i, j, s, r, f, v_{d}} \times \\ \text{EPU}_{i, v_{d}}, \forall i \in M_{s}, f\qquad\qquad\qquad\qquad\quad\ \ \tag{A21}\\ \text{ES}_{i, f}= \sum\limits_{f=1}^{F} \sum\limits_{s=1}^{m} \sum\limits_{i \in M_{s}} \sum\limits_{r=1}^{n_{k}-1}(\text{SM}_{i, r+1, f}-\text{CM}_{i, r, f}) \times \text{ESU}_{i} \tag{A22}\\ \text{ET}_{i, f}= \sum\limits_{f=1}^{F} \sum\limits_{s=1}^{m} \sum\limits_{i \in M_{s}} \sum\limits_{j=1}^{n} \sum\limits_{q=1 \wedge q \neq j} \sum\limits_{r=1}^{n_{k}-1}(U_{i, j, s, r+1, f} \times \\ \text{ETU}_{i, j, q})+ \sum\limits_{f=1}^{F} \sum\limits_{s=1}^{m} \sum\limits_{i \in M_{s}} \sum\limits_{j=1}^{n} U_{i, j, s, 1, f} \times \text{ETU}_{i, j, J_{0}} \tag{A23}\\ \text{TEC}= \sum\limits_{f=1}^{F} \sum\limits_{s=1}^{m} \sum\limits_{i=1}^{l_{f, s}}(\text{EP}_{i, f}+\text{ES}_{i, f}+\text{ET}_{i, f}) \tag{A24}\\ \text{SM}_{i, 1, f} \geqslant \text{ST}_{i, j, J_{0}}-H \times(1- Y_{i, j, s, 1, f, v_{d}}), \\ \forall f, s, i \in M_{s}, j, d \qquad\qquad\qquad\qquad\qquad \tag{A25}\\ \sum\limits_{j=1 \wedge j \neq q}^{n} \text{ST}_{i, j, q} \times Y_{i, j, s, r, f, v_{d}} \leqslant \text{SM}_{i, r+1, f}-\qquad \\ \qquad\ \text{CM}_{i, r, f}+H \times(1- Y_{i, j, s, r+1, f, v_{d}}), \forall j, s, f, i \in M_{s}, \\ 1 \leqslant r< n_{r}-1, d \qquad\qquad\qquad\qquad\qquad\quad\tag{A26}\\ \quad \sum\limits_{d=1}^{n_{d, i}} Y_{i, j, s, r, f, v_{d}}= U_{i, j, s, r, f}, \forall i \in M_{s}, r, j, s, f\tag{A27}\end{gather*}$$ where Formula (A1) makes both makespan and TEC minimized at the same time. Formula (A2) defines the makespan, which should be longer than the completion time of each factory. For each machining position on the machine, Formula (A3) ensures that the start time is greater than 0, and Formula (A4) ensures that the completion time is longer than the start time. For each operation of the job, Formula (A5) guarantees that the start time is longer than 0, and Formula (A6) guarantees that the completion time is longer than the start time. Formulas (A7) and (A8) ensure that two consecutive machining positions on each machine do not overlap. Equation (A9) guarantees that processing will not be interrupted at each position on each machine. Formula (A10) guarantees that for two consecutive operations of the same job, the start time of the successor operation must be after the completion time of the predecessor operation. Equations (A11) and (A12) ensure that only a factory, a machine, a machining position, and a speed can be selected for each operation. Equation (A13) indicates that each operation of the same job should be performed in one factory. Formulas (A14) and (A15) guarantee that the 1st available processing position on the machine should be selected for each operation. Formulas (A16)–(A19) created the relationship between the machine processing position and the assigned operation. Equation (A20) ensures that each job can only be processed in one factory, with no transfers. Equations (A21)–(A23) are three energy consumption calculations, and Eq. (A24) is the total energy consumption calculation. Formulas (A25) and (A26) ensure that setup time should be considered and included in the calculation. Equation (A27) ensures that there is only one speed for each operation at each position on each machine.

Table A1 Statistics on the means of HV metrics in all instances for all variants of the algorithm.

Table A2 Statistical results of all comparison algorithms for the means of HV metrics in all instances.

Table A3 Statistics on the means of GD metrics in all instances for all variants of the algorithm.

Table A4 Statistics on the means of Spread metrics in all instances for all variants of the algorithm.

Table A5 Statistical results of all comparison algorithms for the means of GD metrics in all instances.

Table A6 Statistical results of all comparison algorithms for the means of Spread metrics in all instances.