Development of a Digital Twin Pipeline for Interactive Scientific Simulation and Mixed Reality Visualization

An interactive scientific simulation and visualization pipeline in mixed reality was developed to explore how scientific simulation could proceed in coming workplace. The main technical concern is fast scientific simulator and visualizer in mixed reality with hand gesture. Framework for scientific simulation and visualization was implemented on the server side based on deep learning based fast simulation model and open source visualization toolkit, VTK, respectively. Airfoil aerodynamics was selected as a target scientific simulation task since it is a representative and critical problem in scientific fluid simulation determining the whole aero-product’s performance and shows highly nonlinear physics even by small shape deviation and so high computational cost. Therefore, the fast simulator was firstly developed by carefully developing convolutional neural network-based model, using U-net, which learned corresponding high accuracy simulation dataset. All scenes which are supposed to be seen on the mobile device side were authored in a widely used content development tool called Unity3D with a mixed reality toolkit, MRTK for Microsoft Hololens 2. User interface and user experience were created based on a scenario of a drone rotor which demonstrates running and analyzing the simulation results using designed interaction gestures. Finally, several discussions were derived through the implemented digital twin pipeline. It was concluded, with an adequately implemented mixed reality tools, that scientific researchers or industrial engineers will be helped not only in individual analysis works due to the delicacy of hands and fingers and real time interaction in mixed reality, but also, as a future study, in cooperative analysis works.