Quantifying Variability in Microscopy Image Analyses for COVID-19 Drug Discovery

Microscopy image-based measurement variability in high-throughput imaging experiments for biological drug discoveries, such as COVID-19 therapies was addressed in this study. Variability of measurements came from (1) computational approaches (methods), (2) implementations of methods, (3) parameter settings, (4) chaining methods into workflows, and (5) stabilities of floating-point arithmetic on diverse hardware. Measurement variability was addressed by (a) introducing interoperability between algorithms, (b) enforcing automated capture of computational provenance and parameter settings, and (c) quantifying multiple sources of variabilities for 10 nucleus measurements, from 8 workflow streams, executed in 2 workflow graph configurations, on 2 computational hardware platforms at 2 locations. Using modified Mean Absolute Error (mMAE [%]) to compare measurements, We concluded that for the task of image-based nucleus measurements the variability sources were (1) implementations (0.10 % - 5.72 % per measurement), (2) methods (3.08 % - 3.11 % between Otsu thresholding and CellPose segmentation), (3) parameters (1.16 %-1.17 % between 4- and 8-neighbor connectivity), (4) workflow graph construction and computer hardware (negligible).