Parallel Performance Analysis and Numerical Simulation of Magnetic Nanoparticles Transportation and Blood Flow Pattern in Capillaries

A hybrid CPU-GPU approach is used to investigate the patterns of blood flow and magnetic particles numerically in a capillary under the existence of constant magnetic field. However, the blood flow is considered to be Newtonian, laminar and incompressible in the capillary and magnetic Nanoparticles are assumed as potential agent carrier being used therapeutically for the magnetic targeted drug transport in the fight against diseased cells. The magnetic field is embedded in the muscular volume and is produced by a permanent magnet. The flow field and particles transport dynamics are formulated by a mathematical model and is solved numerically. Finite element discretization gives a big sparse system of equations which needs a higher computation. Therefore, the hybrid approach deals the extensive computations in parallel, as this is a good platform which can decrease the times of solution significantly, if compared to the CPU application. This serves for highly effective search for distinctive mathematical model, along with their distinctive parameters. The impact of pressure <inline-formula> <tex-math notation="LaTeX">$P$ </tex-math></inline-formula>, the capillary radius <inline-formula> <tex-math notation="LaTeX">$R$ </tex-math></inline-formula>; the magnetic nanoparticle radius <inline-formula> <tex-math notation="LaTeX">$R_{M}$ </tex-math></inline-formula>, the magnetic field intensity <inline-formula> <tex-math notation="LaTeX">$H$ </tex-math></inline-formula> on the blood flow along with magnetic particles is studied with regard to the inputs and model of magnetic field. The mathematical results for both the velocity of particles and blood are calculated. It is observed that magnetic field is directly proportional to the flow pattern as an increase in the former upsurge the latter. Moreover, the distance between magnet and capillary wall exerts a direct influence on velocity and is helpful to pull magnetic nanoparticles to the capillary wall. Thus, it is concluded by simulation that the magnetic parameters govern the velocity profile.


I. INTRODUCTION
M agnetic Drug Targeting (MDT) is a significant proposed method for handling cancer like localized abscesses. The drug propagates through the cardiovascular system in non-targeted therapy of drug, directing little concentrations in the abnormal tissue while showing adverse effects in normal tissue. MDT struggles to resolve this issue non-invasively by steering magnetically the therapeutic particles to the targeted The associate editor coordinating the review of this manuscript and approving it for publication was Weipeng Jing . place. A perfect MDT therapy starts either attaching the therapeutic chemically to magnetic particles or entering in a drug carrier. The magnetic therapeutic particles (drug) are then be inserted into the bloodstream at a suitable position while outwardly-applied magnetics field directs them towards the unhealthy tissue, and are forced into the capillary bed of the diseased site, where these particles could be triggered by pH, temperature, magnetic trigger or an enzyme [1]. Regrettably, the above process is challenging partially as the behaviour of magnetic particles is not well understood in blood. VOLUME 8, 2020 This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ A significant research has already been added in literature to solve key problems of magnetic drug directing. The up going potential of the drug directing through the blood flow by magnetic therapeutic carriers demonstrate specifically by [2]- [4]. Several MDT research works apply permanent magnets for particle targeting and capture owing to their strength, cheapness and easy availability.
Both in vitro and in vivo findings have expressed the effectiveness of permanent magnets in catching magnetic therapeutic particles if the magnet is positioned near to the target [5]. The drops of Ferrofluid, according to other studies, can be directed in the same manner [6]. A research has also been carried out to measure the orientation and size of permanent magnets to control particle by applying Halbach array [7] along with the coordination of an outwardly-applied magnetics force with a ferromagnetic wire. Despite of the progress and wide exploration in controlling magnetic particles with permanent magnets, several disadvantages have been noted of their usage. Firstly, short range nature of permanent magnets works has a limited region of effect in the capillary wall, so the magnetic force cannot govern particles deeply inside the body. Then, the physical movement of magnet is the only method for fixing the magnetic field of a permanent magnet in order to be changed at a certain point in space. Electromagnet field is used in many adaptable techniques because electromagnet force can be governed easily by regulating the electric current in order to direct the flow of particles. Experimental studies have shown that modern electromagnet technology can control magnetic micro particles feasibly. However, it is unrealistic to control nanoparticles, that have low potential to stop human capillaries having micron-sized [8], [9] owing to irresistible drag force of fluid. Lately, several papers have been publicized on electromagnet forces, designed specifically for MDT that can create very high magnetic force and gradient in a space having narrow region [10]- [13].
A Lagrangian simulation of magnetic particles in a narrow cylindrical tube under the impact of drag and magnetic forces experimented by [14]. The groups of magnetic particles in shear-thinning and Newtonian fluids are examined by [15], Bandyopadhyay et al. tried out a 2D model to measure the dynamic of particles with magnetic field subjected perpendicular to the streamwise way [16]. David, Cole et al.also formulated two dimensional model to investigate the motion of the magnetic particles expressing buoyancy, magnetic force, gravity and hydrodynamic drag [17]. Moreover, A solitary magnetic particles influenced by gravity, Stokes drag, magnetic force, and hydrodynamic lift in a vertical cylindrical tube framed by [18]. Kumar et al looked the problem of Cu and Al 2 O 3 nanoparticles among two coaxial rotating disks. They obtained the results using hp-Galerkin finite element analysis [19]. Finally, [20], [25] explores blood flow in arteries involving magnetic force by implementing Finite Element Simulation. Many scientific problems are dealt by complex computation and large sparse matrices. For this reason, parallelism has become a new approach to deal the extensive calculations. The microprocessor producers lay stress on increasing cores instead of enhancing single thread performance. Among the several parallel architectures GPUs are the most recognised many core architectures while CPUs are the most shared multi cores architectures [21]. GPUs are used in scientific computing for accelerating intensive and intrinsic parallel computations, as GPUs offer extensive parallelism as compared to CPUs [22], [24]. In addition, the GPUs are preferred over the other architectures in order to speed up a part or the whole of programming code, as its demand is well recognized in the computing market.
A number of studies focus fully on porting the FEM subroutines to be completed by employing GPUs [21], [25]. Many researches concentrate on employing some portions of finite element program on GPUs. For instance, matrix assembly and numerical integration [26], [27], global finite element matrices and the solution of large sparse systems of equation solved by using GPU [28]- [30]. Some previous studies speed up the assembly of large sparse finite element matrix [31], extensive memory of GPU is needed for such approaches. Therefore, it signifies, the problems having huge size which need more memory and it is fulfilling by adding GPUs. For this aim, a hybrid system (CPU/GPU) is needed which has huge memory. Moreover, parallel performance capabilities of CPU/GPU are explored by applying FEM and MATLAB 2018 on a mathematical model.

II. FORMULATION AND DESCRIPTION OF THE PROBLEM
MDT is one of the important ways of drug directing as it helps to concentrate magnetic particles at the targeted place with the help of magnetic force, because, the drug concentration is enhanced at the target which reduces the side effects and toxicity in normal tissue. Tzirtzilakis observed the magnetic force over the magnetic nanoparticles [32]: where the magnetic susceptibility of the particle is χ , volume of magnetic nanoparticles is V M , the magnetic flux density is B and the permeability of free space is µ 0 . The magneto static field have the Maxwell equations as where, − → M = χ − → H the magnetization feature of bio fluid and ∇ is the gradient operator. It is supposed that outwardly-applied magnetics field is nearly invariable for very small particles if measured on the scale of particles. Furthermore, M may be replaced by the magnetization induced in a constant external field. So, the magnetic field across a capillary is supposed to be constant due to the very small diameter of the blood capillary in targeting range. Counteracting forces on the particle in the blood stream within the magnetic field are owing to the flow of blood and can be measured by applying stokes' expression for the drag force on a sphere as: Wherein µ shows the blood's viscosity, radius of the magnetic nanoparticle is showed by R M , u and v represent the velocity of blood and particles. This study deals specifically only those parts of blood capillaries whose orientation is perpendicular towards magnetization. The strength of the magnetic force for the cylindrical magnet in the direction of magnetization is as under: wherein C 1 = 500000 Am − 1 and a is the radius of cylindrical magnet perpendicularly oriented magnetization to the direction of blood flowing in the targeted site as shown in Fig. 1.

III. MATHEMATICAL MODEL
The momentum and mass conservation equations are the leading equations for particles and flow of blood. It is taken for granted that the flow occurs under the effect of outwardly-applied magnetics field towards axial direction. The leading equations of model might be codified in the cylindrical coordinates (r, z, θ) by this assumption. The flux in the blood vessel is supposed Hagen-Poiseulle and the electric conductivity of blood is not viewed. Blood is considered in this research as laminar axisymmetric flux of Newtonian fluid in the capillary having the quality of a viscosity, homogeneity, and incompressibility [3], [33]. The mass and momentum equations are defined as: wherein v is expressed as velocity of magnetic nanoparticles and u shows the velocity of blood. ρ is the blood viscosity and p stands for pressure, N expresses the numeral density of suspended particles, A is stoke,s coefficient, whereas m and t are mass of particles and time parameter respectively.
The independency of pressure gradient is expressed on radial coordinate which appears in (6) eventually, and it is taken from [34].
wherein A 0 stands for constant amplitude of pressure gradient and A 1 shows the amplitude of the pulsatile component which accelerates diastolic and systolic pressure of blood. The model conditions are defined as: at r = R Here radius of the Capillary is R.

A. NONDIMENSIONALIZATION OF EQUATIONS
Nondimensionalization of equations reduce the computational complication and hence the use of following transformation equations rescales the mathematical model.
where H 0 is the power of magnetic field at the magnet's surface and ν = µ ρ is kinematic viscosity. If Transformation equation (9) is applied and by dropping bar, the governing equations (5-6) are written as: where,

IV. NUMERICAL METHOD
The Finite Element Method is a strong technique which is developed originally to solve complex problems in mechanics numerically. In the FEM, the physical model is divided by a set of suitable finite elements interrelated at each point called mesh [35]. Each element is approximated by appropriate polynomial functions. The weak form of equation (8) and (9) over the element e is obtained by the standard procedure. Multiplying equation (8) and (9) with the weight functions W and integrate over the element.
The semi discrete FEM equations (12)(13) are attained by replacing a finite element approximation for the dependent variables, u and v. In selecting the approximation for u and v, it is speculated that time dependence can be detached from the space variation.
Equation (18) Applying the finite element approximation of the form for time derivative as: Using equation (21) in equation (19)(20), the fully developed FEM model is obtained in the form of following equations (22)(23).
Matrix form of equation (22-23) is Known data, secondary variables and interpolation functions are used to calculate the matrices L 11

V. NUMERICAL SIMULATION, RESULTS AND DISCUSSION
This section explores the outcomes of diverse parameters on magnetic nanoparticles and velocity profile of blood along with their numerical results. The table 1 represents the specific values of constants and variables [36]. FEM is used to calculate the velocity trends of blood and particles and computed data is showed for diverse time values, pressure gradient ∇P, magnetization M , magnetic field intensity H and distance (y) between the capillary surface and the magnet. Fig. 2 represents the impact of different time values on axial velocity profiles of blood and particles. Altogether all velocity profiles are similar when they display decline from their maximums at the axis as one goes away from the axis and reaches to zero on the wall surface. Moreover, Fig. 2 indicate pulsatile dependence of blood and particle velocity with time and gains highest velocity at t = 0.8s Trends of blood velocity is showed concomitant with the velocity graph of particles in simulation, and it is anticipated as the blood velocity directs the particles to the targeted location. However, the velocity of particles is noted less than the blood velocity because of the drag and other retardation forces. The upper limit of blood velocity is noted to be 0.14 at the axis and velocity of particles is found to be 0.068. In addition to this, variation from 0.05 to 0.15 cm/sec of velocity of blood in capillary is noted by experiments which agrees with the results given in graphs. Finally, magnetic nanoparticles, it is showed in graphs, travel a very successful journey to the targeted location along with blood to transport the drug.    3 displays that the radius of magnetic nanoparticles and capillary plays a fundamental role in velocity profile and curves demonstrate that velocity of flow rises by reducing the radius of capillary.It is known that if the rate of particle preservation rises up the gradient strength of magnetic field and particle diameter also increase. Particles maximum velocities show the difference of 0.0003 in the case of variation of capillary radius from 3*10-4 to 3*10-6. The noticeable parameter is that manage flow rate can be under the application of magnetic field. The effect of this fact on velocity of nanoparticles is confirmed in Fig. 4. The graph demonstrates that if the value of M is doubled and tripled, the velocity of magnetic nanoparticles rises up to 0.0013 which is highly fit for fatty lump to be break up from the wall of capillary. These graphs indicate that super power magnetic field up rises the velocity remarkably. The distance between magnet and capillary wall exerts a direct influence on velocity profiles and is helpful to pull magnetic nanoparticles to the capillary wall.   5 and 6 depicts variation in velocity of blood and particles with axial distances for different values of pressure gradient, which shows that velocity profiles increase as the pressure increases. It is also noted, when pressure is zero the velocity of blood is near about zero but the nanoparticles have some velocity due to magnetization of particle as shown in Fig. 6. It is clear from graphical results that velocity profiles may be controlled by pressure gradient.
When the distance between magnet and the targeted place is short, the velocity of particles through which these particles travel backward to the magnetic region will be greater as shown in Fig. 7. Magnetic particles velocity varies from 0.00133 to 0.001319 from distance 2.5cm to 10cm. Velocity VOLUME 8, 2020  profiles may be governed by fixing distance between magnet and targeted place, as is indicated in plots, as the strength of magnetic field and gradients depends on the distance between magnet and targeted place.

VI. PARALLEL RESULTS AND DISCUSSION
The key objective in this section is to compare the efficiency of CPU and CPU/GPU based findings of parallel algorithm with computational accuracy. The sequential and parallel codes are settled with the help of Matlab 2018a and simulations are made by a computer having 64-bit operating system, Intel(R) Xeon(R) CPU E5-2620 v2@2.10 GHz, 24 cores and Windows 10. NVIDIA Tesla K20c and NVIDIA Quadro K4000 are two GPU cards which the system possesses. Both cards have 3.5 and 3.0 computing abilities respectively. The NVIDIA Tesla K20c and CUDA cores of NVIDIA Quadro K4000 GPU cards are 2496 and 768 respectively with three GB memory. The clock rate in memory for above cards is 2600 Mhz and 2808 Mhz with 134 GB/sec memory bandwidth. Fig. 8 shows the hybrid algorithm for calculation of data using FEM.
The run time of serial code on CPU is compared with run time of its respective hybrid algorithm to analyse the improvements by CPU/GPU. The numerical results to measure the CPU performance evaluation for all test cases is shown in Table 2. The first case has 12000, 2nd has 18000 and third possesses 25000 number of elements while the unknowns are 24002, 36002 and 50002 individually. Direct proportion of matrix size is also showed by table 2, the execution time of CPU also rises as the size of matrix increases.  Fig. 9 shows CPU-GPU results of numerical schemes for the above three tests. As shown above, the tests are executed using 1200, 1800 and 2500 number of finite elements for large sparse matrices while the corresponding number of unknowns are numbered 24002, 36002 and 50002 individually. Set-up times and initialization are not included from the time measurements and the detail solution is made by storing data at each time iteration. The number of threads for every block is selected from 1-512 for kernel execution. Fig. 9 shows the time of execution for each test. The time of execution comes down when the number of threads for every block rises up to 128, later on, the time of execution rises along the number of threads for every block and it may occur due to communication overhead process.
The speedup and temporal performance (TP) are significant indicators to gauge the performance of CPU/GPU platform. This platform is supposed to be the optimum if it is executed in lowest run time. The ratio of execution times of CPU/GPU and CPU is known as speedup whereas, the converse of the execution time gives temporal performance. Temporal performance and speedups are attained by the GPU with respect to threads with range 1-512. These indicators demonstrate the performance of CPU-GPU as shown in Fig. 10    number of threads, due to overhead inter-process communication. At that point, the increase in number of threads does not enhance the speedup, but it will only add more overhead. These increments exhibit the CPU-GPU performance improved in terms of speedup. From another point of view, the temporal performance for large domain size is higher as compared to small domain sizes. The maximum value of TP exhibits that the computations are measured in smallest possible time. The performance related to time increases  from thread 1-128, afterwards, it drops-off owing to overhead communication. The higher value of TP for sparse matrix depicts that GPUs have the capability of dealing large sparse computations.
The efficiency and effectiveness are another important indicator to evaluate the results of CPU-GPU performance as shown in Fig. 11 for all cases. The effectiveness drops-off VOLUME 8, 2020 as number of threads rises owing to deadlock, process congestion and overhead in all cases. But it is noted that the effectiveness for large size of domain is higher as compared to small size of domain which shows that CPU-GPU may efficiently deal extensive calculations. The parallel results clarify that CPU-GPU can deal large data more competently than CPUs. Other PPI demonstrates better results for large size of domain. Consequently, when the domain size is big the GPUs are able to accelerate the parallel performance results. These results verify that in large scale computational simulations, the use of GPUs is better.

VII. CONCLUSION
The present theoretical exploration explores the effect of outwardly-applied magnetics field on the magnetic nanoparticles and several aspects of blood flow. The key findings conclude that outwardly-applied magnetics field can control satisfactorily the velocity of magnetic nanoparticles and blood. Another possibility is noted that these quantities can come down to any required level by decreasing or increasing the intensity of magnetic force and other parametric values. The physical parameters also exert a prominent influence on the flux of blood in capillary. The interconnected mutual contribution between simulation and modelling is highly beneficial to govern the targeted location more precisely with the intention to obtain improved results of drug delivery through blood in capillary. Moreover, this study describes our inspiration for using CPU-GPU accelerators for handling computational complications. The numerical solutions are obtained by using hybrid platform and gained reasonable parallel performance results such as time related performance and speed ups. The results show that a hybrid solver is more effective in extensive calculations. The performance results have also exhibited accelerations in calculations by using GPUs devices. Hence, numerical results demonstrate that parallel approach in the finite element permits to decrease simulation time. The speed-ups are encouraging despite of the fact that the GPU is used only in some parts of the program, which encourages us to advance in this track.
AKHTAR ALI received the M.S. degree in computational mathematics from the University of Engineering and Technology Lahore, Pakistan, in 2011, and the Ph.D. degree in computational mathematics from the University of Technology Malaysia, in 2019. He is currently an Assistant Professor with Government College University Faisalabad, Pakistan. His research interests include mathematical modeling, parallel computing, and scientific computing.
MAJID HUSSAIN received the master's and Ph.D. degrees from Quaid-i-Azam University Islamabad, Pakistan. He is currently an Assistant Professor and the Head of the Department of Natural Sciences, Humanities and Islamic studies, University of Engineering and Technology (RCET), Lahore. His research interests include fluid dynamics, biomechanics, numerical analysis, applied mathematics, nanofluid, and heat and mass transfer.