A Multi-Server Queuing Model With Balking and Correlated Reneging With Application in Health Care Management

Queues or waiting lines are an integral part of health care facilities such as hospitals, outpatient clinics, medical laboratories, and many other health facilities. Health care management must have waiting lines control strategies for smooth functioning. Due to the lack of proper queuing control and management, patients may become dissatisfied and may leave (renege) the health care facilities without getting service. But, the reneging of patients at two consecutive time marks may be correlated in the sense that if a patient reneges at the current time mark, then there is a probability that a patient may or may not renege at the next time mark. This kind of reneging is referred to as correlated reneging. In this paper, we have introduced the concept of correlated reneging in a finite capacity multi-server queuing model with balking with its application in health care. The steady-state as well as the transient analyses of the model are carried out. We have also derived an expression for the correlation coefficient between the inter-reneging times and for the rate at which the health facility is losing patients (patient loss probability) due to insufficient capacity, reneging, and balking. We have provided numerical examples in order to demonstrate the effect of balking and correlated reneging on performance measures such as the mean number of patients waiting to be serviced, mean waiting time of patients, and the probability of patient rejection. Further, the effect of the number of servers on performance measures is investigated. Finally, the effect of the correlation coefficient between the inter-reneging times on performance measures is studied. The queuing model discussed in this paper could be useful to the health care firms facing the problem of patient impatience and capacity constraints.


I. INTRODUCTION
Queues or waiting lines are ubiquitous and sometimes unavoidable in health facilities such as outpatient clinics, medical laboratories, dentists, and many other health facilities. Coupled with the increasing demand for health care services due to the growing population especially in developing countries, the number of patients waiting to receive service at a given time will continue to grow, resulting in long waiting The associate editor coordinating the review of this manuscript and approving it for publication was Rashid Mehmood . times and saturation of the facilities. In order to improve the quality of service in health facilities, it is essential to allocate sufficient resources to mitigate the effects of long waiting times on the behavior of patients [24], and a trade-off must be made between improving the quality of service and the optimal use of resources at the health facility.
The waiting lines in health facilities can be modelled as queuing systems in which patients arrive, join a waiting line if any, receive service at their turn and then depart from that resource. Fomundam and Herrmann [10] surveyed the contributions and applications of queuing theory in the VOLUME 8, 2020 This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ design and analysis of health care systems. Lakshmi and Shivakumar [13] presented a review of the applications of queuing theory in health care management, which includes the use in the design and the analysis of health care systems and operations. The authors indicated that the performance measures that should be considered when evaluating health care systems should include waiting times in queues, resource utilization, and rejection of patients when the maximum capacity of space in the waiting line is reached.
The majority of the queuing models proposed for health care systems often assume that when a patient arrives at the queuing facility, the patient must join the queue and remain in the queue until the patient receives service. A patient that arrives and observes a long queue may decide not to join the queue (patient balking), but if the patient decides to join the queue and experiences a long wait or is dissatisfied with the health care services, the patient may decide to leave the queue without receiving service (patient reneging). The authors in [31] presented a queuing theory-based method for the estimation of the fraction of patients that leave a hospital Emergency Department (ED) without treatment by considering balking and reneging of patients. They studied the relationship between the arrival rate and the patient impatience (balking and reneging) but did not investigate the influence of balking or reneging on health care performance measures such as the number of patients waiting in the queue, the average waiting time and the probability of patient rejection when the health care facility is saturated. Also, during epidemic and pandemic periods such as the current coronavirus disease 2019 (COVID- 19), health facilities are overwhelmed and may decide to postpone the treatment of patients with mild cases, that is, they may be removed from the queue without being served. Balking is a situation where a customer arrives and decides not to join the waiting line while reneging is a situation where a customer becomes dissatisfied and leaves the waiting line without receiving service [23].
The work done so far in the area of queuing models with impatient customers, reneging was considered as a function of the state of the system (such as queue length or time in the queue). But, the reneging could be bursty in nature based on exogenous factors in many practical situations. That is, the reneging may depend on factors other than the system state. We have many practically visible situations like reputed health care systems, purchase of branded products, admission in prestigious schools, where the length of the queue does not discourage the customers in the queue. In such high-end products and services, customers like to stay in the queue. But, the sustainability of these products and services in the market depends upon the perception in the masses. The perception created influences the decision-making of customers where similar customers arrive together (physically or virtually). If the perception goes wrong about the product or service, word-of mouth publicity severely prompts the customers to renege (i.e. to abandon the system) before getting the service. For example, let us consider a health care system where the arrival of patients for the check-up is similar to the arrival of customers, the check-up of patients by a doctor is analogous to the service of customers, and the leaving of the patient from the health care system before being checked-up can be considered as reneging of customers. The reneging of patients could be bursty at times because of the reasons like improper diagnosis/treatment, unnecessary costly prescription by doctors, etc. which may create a bad perception among masses. That is, if a patient reneges at any time instant, then there would be an increased probability of a patient to renege at the next time instant without being checked-up. Thus, the probability of reneging is based on a recent customer reneging where similar customers arrive close together to exchange their views. So, if a patient observes that an earlier patient reneges, then influenced by his decision and opinion he may also decide to renege. This kind of reneging is referred to as correlated reneging. Sometimes it happens that the arriving patient does not join the health care system. This situation resembles with the balking in queuing theory.
Most of the studies of correlated queues are based on steady-state analysis only. The steady-state analysis gives the estimates of the performance measures over long term observation when the system becomes stable. But in reality, to know about the system-state up to some instant t will be more useful. There are many systems that start operations and then closed at some specified time t. Such systems like health care facilities, businesses, or service operations that open and close, never operate under steady-state conditions. Moreover, if at the initial time the system is empty, the fraction of time the server is busy and the initial rate of output, etc., will be below the steady-state values so as to use the steady-state results for calculating performance measures is not justified. Thus, the analysis of the transient behavior of queuing systems is very necessary from the application viewpoint.
In this paper, we propose and analyze a queuing model with balking and correlated reneging of patients for the performance evaluation of a health care system. In the queuing literature, the reneging considered so far is dependent only on the system state. But, reneging may depend on factors other than the system state. We have taken this idea into consideration and developed a multi-server queuing model with balking and correlated reneging. The main contribution of this paper is the concept of correlated reneging, and the analytical investigation of the impact of the correlation coefficient between inter-reneging times on the performance measures. We have solved the model both in transient and steady-state. The rest of the paper is arranged as follows: in section 2 literature review is presented. In section 3, the queuing model is described. In section 4, we derive the formula for the correlation coefficient between inter-reneging times. In section 5 the mathematical model is presented, in section 6 and 7 the steady-state and transient-state analysis of the model are discussed respectively, and finally, the paper is concluded in section 8.

II. REVIEW OF RELATED LITERATURE
The studies on the queuing models with customers' impatience started in the early 1950s. Haight [12] incorporated the concept of balking in a single server queuing model. An arriving customer joins the queue if the number in the queue is less than the greatest queue length that he will tolerate. Haight [11] studied a single server queuing system where a customer joins the queue, waits for service, and may decide to leave the queue without receiving service if his waits exceed his maximum expected wait. He performed the steady-state analysis of the model. Ancker and Gafarian [4] studied an M/M/1/N queuing model with balking and reneging. Arriving customers balk with probability n/N , where n is the number of customers in the system and N is the capacity of the system. Customers reneged according to an exponential distribution. They studied various measures of performance. Anker and Gafarian [5] considered an infinite capacity Markovian single server queuing model with balking and reneging. In this paper, the inter-arrival times, service times, and the reneging times all were exponentially distributed. The arriving customers joined the queuing system if it is empty or balked with probability 1 − (β/n); n = 1, 2, 3, . . . where n is the number of customers in the system and β is a measure of customer's willingness to join the queue. Subba Rao [25] studied a finite capacity M/G/1 queuing system with balking, reneging, and interruptions. The serving of the customers is subject to breakdowns caused by the arrival of interruptions that have to be cleared on a priority basis. The supplementary variable technique and discrete transforms were used to obtain the solution of the model. Since then, a number of researchers have worked on various queuing models with reneging and balking. Choudhury [6] presented the analysis of a queuing system in which customers tend to become discouraged and give up when the waiting time exceeds a random threshold. The authors assumed that the random thresholds are independent and identically distributed exponential random variables. Wang et al. [28] considered a reneging process in which customers leave the queue without being served and the reneging times were assumed to follow a negative exponential distribution. Wang et al. [29] presented a survey of queuing systems with impatient customers in which the authors discussed impatient behaviors of queuing systems such as balking and reneging and also discussed their analytical solutions, numerical solutions, and simulations. The transient solution of multi-server Markovian queuing system (M/M/c) with balking and reneging was proposed in [3] where after joining the queue each customer has to wait for a certain length of time T which has exponential distribution for service to begin. Kumar et al. [15] studied the transient solution of a single-server infinite capacity Markovian queuing system with balking. Recently, Kumar and Sharma [16] introduced the new concept being referred to as the retention of reneging customers in queuing theory. They have studied an M/M/1/N queuing system with reneging and retention of reneging customers. They have obtained the steady-state solution of the model. Kumar and Sharma [14] considered a single-server queuing model with reneging and retention of reneging customers. They have derived the transient solution of the model. Kumar and Sharma [17] obtained the transient solution of an M/M/c queuing model with balking, reneging, and retention of reneging customers. [30] incorporated the concept of retention of reneging customers in a finite capacity queuing system with working breakdowns. They obtained the steady-state solutions using the matrix-decomposition method. Kumar and Soodan [18] studied the transient behavior of a single-server queuing model with correlated arrivals and reneging numerically.
Mohan [19] was the first to introduce the concept of correlation in gambler's ruin problem. Murari [21] studied a queuing system with correlated arrivals and general service time distribution. Mohan and Murari [20] obtained the transient solution of a queuing model with correlated arrivals and variable service capacity. Conolly [8] considered a queuing system having services depending on inter-arrival times. Conolly and Hadidi [9] considered a model having an arrival pattern impacting the service pattern. They examined the initial busy period, state and output processes. Cidon et al. [7] considered a queue in which service time is correlated to inter-arrival time. They studied this correlation in the case of communication systems and showed the impact through numerical results by comparing them with less reliable models.
Queuing theory can be used to provide a reasonably accurate approach to size and evaluate the performance of health care facilities. Performance evaluation of health care systems partly involves the use of queuing theory and other stochastic modelling techniques to estimate the performance measures such as the average time spent by patients in waiting lines, utilization of the resources, the losses due to reneging, balking and rejection of patients, and the probability of immediate service. Patient's satisfaction surveys presented in [27] show that the majority of complaints from patients are usually related to long waiting times, sometimes due to insufficient resources, and insufficient waiting room. The authors in [2] modelled the performance of a health facility using a queuing network analyzer and discrete event simulation. Obulor and Eke [22] applied queuing theory for the evaluation of an outpatient appointment system to model the appointment scheduling process in order to reduce patients' waiting times and the idle times of the health personnel. The authors in [1] discussed recent mathematical modelling techniques and issues in Operations Research in the context of health care system planning using queuing theory.
In the literature discussed so far, no work has appeared on the concept of correlated reneging in queuing theory and its application in health care. Furthermore, we have shown most of the studies related to the application of queuing theory in health care are based on the steady-state analysis. That is why we have presented a queuing model with correlated reneging with its possible applications in health care. The transient as well as steady-state analysis are performed. VOLUME 8, 2020

III. QUEUING MODEL DESCRIPTION
Patients arrive at the health care facility for the check-up and form a queue if the service is not immediately available. In general, the first come, first served queue discipline is used in the scheduling of patients for service, but other service disciplines can also be applied. The time required to service a patient, for example, for consultation, may not be deterministic. Some patients may require a short time to be served while the others may require a very long time, which may sometimes result in longer delays than the patient had estimated in order to plan other activities especially for those with mild health problems. The queuing model under investigation is based on the following assumptions: 1) The patients arrive at a service facility one by one following a Poisson process with parameter λ. 2) On arrival, an incoming patient may decide not to join the queue (i.e. balk) with a certain probability (say, 1 − β). It means that the arriving patient may join the queue with a probability β. 3) The system has a single queue and finitely many servers, c. The service-times at each server are independently, identically, and exponentially distributed with parameter µ. 4) The capacity of the system is finite (e.g, K ). Further, K = N + c, where N denotes the queue capacity and c is the number of servers (health personnel). 5) After joining the queue and waiting for some time, a patient may get impatient and leave the queue without getting the service. The reneging of the patients can take place only at the transition marks t 0 , t 1 , .. That is, the distribution of inter-transition marks is negative exponential with parameter ξ . 6) The reneging at two consecutive transition marks is governed by the following transition probability matrix: to t r 0 1 0 p 00 p 01 from t r−1 1 p 10 p 11 where, 0 refers to non-occurrence of reneging and 1 refers to the occurrence of reneging. Thus, the notation p ij (i and j can either be 0 or 1) represents the probability of transitioning from present state to next possible state due to the reneging between two consecutive transition marks. Also, p 00 +p 01 = 1 and p 10 +p 11 = 1. Thus, the reneging in two consecutive transition marks is correlated. In case of no correlation p 00 = p 10 and p 01 = p 11 .

IV. CORRELATION COEFFICIENT BETWEEN INTER-RENEGING TIMES
Correlation coefficient (ρ) is a quantitative evaluation that measures the direction and strength of variation in one variable when the values of other variable are changed. The correlation coefficient between the inter-reneging times helps us in understanding the extent of reneging between two consecutive transition marks.
Let t r ; r = 1, 2, 3, . . . be a sequence of transition marks (random variables) such that t r takes only two values 0 and 1 with conditional probabilities as given in the following transition probability matrix.
to t r 0 1 0 p 00 p 01 from t r−1 1 p 10 p 11 , where, 0 refers to non occurrence of reneging and 1 refers to the occurrence of reneging. Now, We can obtain p r as: The solution of difference equation (1) is given by: We can obtain the following values: VOLUME 8, 2020 and, where E{t r }, Var{t r }, andCov{t r−1 , t r } are mean, variance of t r and covariance between t r−1 and t r respectively. Hence, correlation coefficient between the inter-reneging times (ρ) is given by: Suppose that X (t) is a random variable that represents the number of patients waiting in the queue, and that P{X (t) = n} = P n,r (t) is the probability that at time t, there are n patients waiting in the queue, where r = 0 indicates that a patient had not reneged at the previous transition mark, and r = 1 implies that a patient had reneged at the previous transition mark. Consider Q 0,r (t) to be the probability that at time t, there is no patient in the queue and all the servers are idle. Also consider R k 0,r (t) to be the probability that at time t, there is no patient in the queue but k, ( 1 ≤ k ≤ c) servers are busy. The difference-differential equations that describe the time-dependent state probabilities for the number of patients in the health care queuing system are: In the next two sections, we will present the steady-state analysis and the transient-state analysis of the above differential equations in order to determine the steady-state and the transient-state performance evaluation measures of the proposed queuing model.

VI. STEADY-STATE ANALYSIS OF THE MODEL
In this section, we obtain the steady-state probabilities of the queuing model by using the matrix-decomposition method.

A. STEADY-STATE EQUATIONS
Let us define the steady-state probabilities of the number of patients in the queuing system as follow: .c, i= 0,1, and P n,i = lim t→∞ P n,i (t), where n=0,1,2,..N is the number of patients and i = 0 implies that a patient had not reneged at the previous transition mark and i = 1 implies that a patient had reneged at the previous transition mark.
Proof: The proof is provided in Appendix-1.

B. STEADY-STATE PERFORMANCE EVALUATION MEASURES
After obtaining the steady-state probabilities by solving the above steady-state equations, we can then obtain the performance measures such as the average number of patients waiting in the queue, the average waiting time of patients in the queue, the rate at which the health facility is losing patients (patient loss probability) due to insufficient capacity, reneging and balking. The average number of patients waiting in the queue is If a patient arrives at the health facility and its capacity has been reached, then the patient will be rejected (not admitted into the queue) with a probability P N ,r , and the instantaneous dropping rate of patients is λ r = λ(P N ,0 + P n,1 ). Also, if a patient arrives and sees n patients waiting in the queue, the probability that the patient does not join the queue (balking) is β, then the instantaneous balking rate is λβ. Therefore, the average balking rate [32] is Suppose that there are n patients in the queue, and any of the n patients in the queue could renege, then the instantaneous reneging rate is nξ . Therefore, the average reneging rate is The average rate at which the health facility is losing patients due to insufficient capacity (patient rejection), balking, and reneging is The probability of losing patients by the health facility (patient loss probability) due to insufficient capacity, reneging, and balking is Since patients that arrive when the capacity of the system has been reached and those that balk (do not actually join the queue), then the effective arrival rate is The steady-state waiting time can be obtained using the Little's law as It is desirable to choose the capacity of the health care facility and to allocate service resources in such a way to reduce the waiting time and patient loss probability.

VII. TRANSIENT ANALYSIS OF THE MODEL
In this section, we study the transient behavior of the model. As the analytical solution in the transient case is quite complicated to obtain, we use a numerical method (Runge-Kutta method of fourth-order) to obtain the transient solution of the model. The ode45 function of MATLAB software is used to compute the transient numerical results.

A. TRANSIENT-STATE PERFORMANCE EVALUATION MEASURES
The transient performance measures considered are: the number of customers in the queue, the waiting time in the queue, balking rate, and the reneging rate. The mean number of patients in the queue at time t, L q (t) and the mean waiting time in the queue estimated at time t, W q (t) are given as (38) and (39), as shown at the bottom of the next page.
Similarly, the average balking rate estimated at time t, R b (t), and the average reneging rate estimated at time t, R r (t) are given by The transient loss rate due to blocking (rejection) is λ r (t) = λ(P N ,0 (t) + P N ,1 (t)) (42) Therefore, the transient patient loss rate is We also compare three different queuing models: queuing model with correlated reneging and balking, queuing model with correlated reneging, and queuing model with simple reneging and balking. We plot the graphs to show the effect of the rate of transition marks and the probability of balking on the performance measures. Figures 3 and 4 show the variation of state probabilities of the number of patients with time. For a given arrival rate of patients, the number of active resources and service rate of each resource, all the state probabilities increase rapidly initially within a short time and then attain steady-state after some time. However, the state probability P 1,0 (t) is initially high because we have assumed that there is one patient at the facility initially, that is, P 1,0 (0) = 1. Figure 5 shows the variation of the mean number of patients in the queue with time. As time progresses the mean number of patients in the queue decreases gradually and becomes steady after some time. The higher values of the mean number of patients initially are due to the initial condition P 18,0 (0) = 1. From figure 6 we can observe that the mean waiting time of patients in queue decreases with time. It becomes steady with the passage of time. The higher  waiting times at the start of the system are due to the higher queue lengths initially, the reason being we have assumed 18 customers in the system at time t=0. From figures 5 and 6 L q (t) = N n=1 n[P n,0 (t) + P n,1 (t)] (38)  we find that the mean queue lengths as well as the mean waiting times of the patients in a queuing model with balking and reneging (exponentially distributed reneging times) are lowest as compared with the queuing models having correlated reneging, and balking and correlated reneging respectively. Further, we can see that the queuing system with correlated reneging possesses higher mean queue lengths and mean waiting times as compared to the queuing system with balking and correlated reneging. In the case of correlated reneging, reneging is not state-dependent. The reneging of patients occurs due to factors other than the system state.
Here the reneging at two consecutive transition marks is probabilistic in nature. That is why, the reneging is not as frequent as in case of simple exponential reneging. Therefore, the mean queue lengths and the mean waiting time are highest in this case. From figure 7, we observe that the probability of patient rejection initially increases to a certain value and then decreases slowly and after some time attains steady state. It can also be seen that the probability of patient rejection is highest in the case of a queuing system with correlated reneging as compared with the other two systems. This happens because the mean queue lengths are higher in the case of a queuing system with correlated reneging as compared to the other systems under consideration. Balking reduces the system size significantly, that is why the probability of patient rejection is smaller in the case of a queuing system with balking and correlated reneging as compared to the queuing system with correlated reneging. The probability of patient rejection is lowest in the case of queuing system with exponential reneging and balking as the mean queue length is lowest in this case as compared to the other two systems. Figures 8, 9, and 10 demonstrate the influence of the number of active resources (servers) on the mean number of patients in the queue, mean waiting of patients, and the probability of patient rejection. In the consultation queue of outpatients, the active resources are medical staff attending to the patients. If we increase the number of active resources (number of medical staff) the mean queue length, waiting time and the probability of patient rejection reduce significantly. However, adding the number of staff attending to patients also increases cost, necessitating the use of analytical VOLUME 8, 2020  tools in planning and sizing health facilities while taking into consideration patient satisfaction. Figures 11, 12, and 13 show the changes in the mean number of patients in the queue, mean waiting of patients in the queue, and the probability of patient rejection with changes in the arrival rates of patients. Different factors may cause the rate at which patients are arriving at a health facility to vary, it could be due to an epidemic outbreak, increase in the reputation of the health facility and quality of service offered to patients and may other factors. We can see from the figures that as  the arrival rate increases, the performance measures increase slowly and at a certain value of arrival rate a very small increase in the arrival rate will cause a large increase in the performance measures. Hence, it may lead to degrade the performance of the health facility because the patients may have to wait for very long hours to receive service or the waiting space at the facility becomes permanently full, leading to persistent rejection of patients. Figure 14 depicts the variation in the mean number of patients with respect to the  rate of transition marks. As we increase the rate of transition marks, the mean number of patients decreases. Figure 15 shows the variation in expected waiting time in queue with respect to the rate of transition marks. One can observe the decrease in mean waiting time with the increase in the rate of transition marks. In figure 16 the variation in the probability of patient rejection with respect to the rate of transition marks is studied. It is noticed that with the increase in the rate of transition marks the probability of patient rejection decreases.   Figure 17 depicts the variation in mean number of patients with respect to the probability of balking. As the probability of balking is increased the mean number of patients decreases. Figure 18 presents the variation in mean waiting time with respect to the probability of balking. One can observe that the expected waiting time decreases with the increase in the probability of balking.
In figures 19 and 20 the impact of the correlation coefficient between inter-reneging times (ρ) on the performance  measures in the mean number of patients in the queue (L q (t)) and mean waiting times of patients in the queue (W q (t)) is investigated. We use equation (2) to calculate the values of correlation coefficient (ρ). For illustration, we fix the values of p 00 = 0.1 and p 01 = 0.9 and vary the value of p 11 from 0.1 to 0.9. From figures 19 and 20 we can observe that as we move from negative to positive correlation, the mean number of patients in the queue as well as the mean waiting time of patients in the queue both decrease. Moving towards a positive correlation between inter-reneging times  means the probability of reneging between inter-reneging times (transition marks) increases, which in turn results in the loss of patients from the health care facility. Although the congestion and mean wait decrease with the increasing values of correlation coefficient between inter-reneging times, the decision-makers must see its negative impact on the revenue generation of health care facilities, because the potential customers (patients) leave the service facility without taking service. Therefore, with such scenarios, the health care service providers should think of avoiding this kind of impatience among the patients by incorporating either more number of servers (staff) or machines for efficient service environment.
But, when we study the change in correlation coefficient between inter-reneging times by changing the value of p 00 and fixing the values of p 01 and p 11 , then we will see that while moving towards the positive correlation the values of performance measures L q (t) and W q (t) will increase.

VIII. CONCLUSION AND FUTURE WORKS
Waiting lines are unavoidable in health facilities. Queuing aspects of patients must be taken into consideration while designing the health care facilities. A finite capacity multi-server queuing model with balking and correlated reneging is studied and analyzed with its potential applications in health care systems. We have also derived an expression for the correlation coefficient between the inter-reneging times and for the rate at which the health facility is losing patients (patient loss probability) due to insufficient capacity, reneging, and balking. The numerical examples discussed in the paper provide useful insights about the functioning of the finite capacity health care system having balking and correlated reneging of patients. Therefore, the capacity of health care facilities and the service resources should be carefully allocated in such a way as to reduce the waiting time of patients in the queues and the patient loss probability within a reasonable cost.
In the future, we will incorporate the concept of correlated reneging in non-Markovian queues. We will also consider the time-dependent arrival and service rates in the current model. An attempt will also be made to study the same model with correlated arrivals. We have presented a theoretical queuing model for health care management in this paper, but a case study can be done on this model with real-time data from health care facilities in the future.
Here, A 16  Putting the value of R 1 from (77) in (75), we get R 1 0,1 = 3 A 65 P 1,0 Substituting the value of P 0 from (74) in (70). We get the value of P 1 as: We can obtain the value of Q 0,0 and Q 0,1 by substituting the value of R 1 0,0 and R 1 0,1 from (69) and (78) in (44) We can obtain the unknown constant P 1,0 by using normalizing equations: = Q 0,0 + R 1 0,0 + R 0 e + P 1,0 + P 0 e +Q 0,1 + R 1 0,1 + R 1 e + P 1,1 + P 1 e = 1 where e is the unit column vector of dimension N. PIOTR CZEKALSKI is currently working as a Researcher and an Educator with the Silesian University of Technology, Gliwice, Poland, and Riga Technical University, Riga, Latvia. His research interests include the Internet of Things, unmanned aerial vehicles, fuzzy logic and fuzzy inference systems, AI, neural networks, and evolutionary computation. He also has a research track on the theoretical and applied finite-state automata. He has over 20 years of experience as a Professional Educator and a Tutor for higher education and commercial companies. He also has broad experience in managing educational and research and development projects and grants, in both university and commerce. His personal interests focus on sports and travel. VOLUME 8, 2020