A Survey on Optimal Transport for Machine Learning: Theory and Applications

Optimal Transport (OT) theory has seen increasing attention from the computer science community due to its potency and relevance in modeling and machine learning (ML). OT provides powerful tools for comparing probability distributions and producing optimal mappings that minimize cost functions. Consequently, OT has been widely implemented in computer vision tasks such as image retrieval, image interpolation, and semantic correspondence, as well as in broader applications spanning domain adaptation, natural language processing, and variational inference. In this survey, we aim to convey the emerging prominence and widespread applications of OT methods across various ML areas and outline future research directions. We first provide a history of OT. We then introduce a mathematical formulation and the prerequisites to understand OT, including Kantorovich duality, entropic regularization, KL Divergence, and Wasserstein barycenters. Given the computational complexity of OT, we discuss entropy-regularized version of computing optimal mappings that facilitate practical applications of OT across diverse ML domains. Further, we review prior studies on OT applications in ML. To this end, we cover the following: computer vision, graph learning, neural architecture search, document representation, domain adaptation, model fusion, medicine, natural language processing, and reinforcement learning. Finally, we outline future research directions and key challenges that could drive the broader integration of OT in ML.