Scanning Electron Microscope Image Segmentation with Foundation AI Vision Model for Nanoparticles in Autonomous Materials Explorations

The role of scientists and engineers is being significantly reshaped by the increased role of artificial intelligence (AI) across various science domains. Materials science is a field that is well-positioned to benefit from the integration of AI in the design of automated experimental systems. This is especially true for systems that are collecting and analyzing massive quantities of data from scanning electron microscopes (SEM). The associated SEM imagery is on the nanoscale, offering unprecedented fidelity to analyze materials in their most basic and simple structures. However, the richness and potential volume of this data necessitate human-machine teaming, specifically AI tools to accelerate scientific discovery. This paper details how the foundation AI vision Segment Anything Model (SAM) is leveraged within an experimental workflow. Specifically, SAM’s basic use requires significant human-in-the-loop effort to mark relevant areas. Herein, we detail how SAM creates excessive segmentation masks and describe the necessary post-processing to refine and filter outputs to be suitable for an autonomous materials experimentation pipeline. Our vision system design that leverages SAM coupled with classical computer vision as a post-processing stage achieves 98–100% precision isolating Aluminum nanoparticles within SEM images.