Machine Learning-Based Real-Time Localization and Automatic Trapping of Multiple Microrobots in Optical Tweezer

Optical Tweezer (OT) is an attractive tool for biological studies, which has been used for cell manipulation and tissue engineering. However, the high intensity of laser beam may damage the target biological objects or specimens. To this end, indirect optical manipulation using optical microrobots has become a promising research direction. To enhance the efficiency of indirect manipulation, automatic localization and trapping of multiple microrobots is significant, which paves a way for closed-loop control. For microrobots localization, a modified YOLOv4-tiny neural network model is developed by integrating ConvNext block for feature extraction, while Gaussian modelling is used to optimize the coordinates of the output of the detection head. To avoid expensive manual annotation for model training, self-supervised learning method is employed with Mosaic data augmentation to eliminate the need of collecting a large amount of data. To determine the optimal trapping points for the laser spots, a machine learning-based ellipse detection method is developed based on arc-support groups and k-means algorithm. We conducted experiments to evaluate the effectiveness of the proposed methods. Compared to mainstream object detection algorithms, our proposed mi-crorobot localization method has higher accuracy and enhanced computational efficiency. The proposed ellipse detection method can detect spherical structures of localized microrobots effectively with 96.77% of success rate. The code is open-source and released on: https://github.com/ClouseYunxiao/MicrorobotsDetection_TrappingPoints