In the field of nanotechnology, tracking freely swimming microorganisms under a microscope is difficult. An image of the target often rotates and changes shape as it moves. Further, the difficulty increases with background changes and fluctuations of colors and the brightness because of differences in the surrounding environment. To address these problems, we propose a tracking method combined with robust image processing of the data obtained by the tracking system. Our system feeds image processing data back, and obtains accurate positions of swimming microorganisms in real-time.