Calibration of Electron Microscopes Through Deep Learning and Bayesian Optimization

Electron microscopes can create images at atomic resolution, enabling key developments across many scientific fields. To obtain high-resolution images, the inherent imperfections in the electromagnetic lenses have to be overcome by frequent and extensive calibration. Currently, this calibration is typically performed or monitored by experts, making it expensive and time-consuming. To resolve this, this paper proposes a data-driven solution for automated, image-based calibration of a Scanning Transmission Electron Microscope (STEM). The method uses deep learning to interpret the high-dimensional image, also called the Ronchigram, produced by the STEM, providing a quality index that quantifies the distance of the current image from its calibrated state. Specifically, we perform transfer learning on an adapted ResNet18 architecture using a large data set of simulated Ronchigrams. Afterward, Gaussian process regression models how calibration settings affect this quality index from limited samples. Subsequently, candidate calibration settings are selected online using Bayesian optimization with a novel acquisition function tailored to this problem. When applied to the STEM, the proposed method consistently calibrates defocus and two-fold astigmatism in just a few steps. Compared to existing automated calibration methods, the proposed method can consistently achieve similar performance in less time. For this reason, our approach potentially reduces expert involvement and improves the overall throughput of the STEM. Moreover, the data-driven nature of the proposed method suggests it could be adapted to calibrate additional types of imperfections, positioning it as a flexible tool for enhancing STEM performance, provided that representative data can be generated from the simulator.