Cross-Modal Contrastive Learning for Robust Representation of the Extracellular Matrix in Static and Dynamic Full-Field OCT Images

The full-field optical coherence tomography imaging system can produce two complementary modalities: static (FFOCT) and dynamic (DCI). FFOCT captures the highly scattering tissue fibers, while DCI leverages an endogenous cell contrast derived from moving scatterers. We learn a common embedding in a cross-modal latent space which generates a robust representation of the extracellular matrix in DCI, where it is subject to several imaging inconsistencies, but is well rendered in FFOCT. In this regard we train a siamese convolutional neural network which learns the cosine distance between feature vectors of pairs of corresponding images. We show that a classification inspired loss implicitly learns a hard margin metric, we also propose two acceptance measures to validate the method: the identity error and the symmetry error, both accounting for errors < 1%.