I. Introduction
The rapidly increasing demand for network traffic data, driven by the advancements in 6G, artificial intelligence (AI), and the Internet of Things (IoT), necessitates continuous research efforts to achieve higher spectral efficiency. In recent years, coherent optical communications have emerged as a promising solution for achieving large capacity and high-speed data transmission over optical fibers. Coherent optical transmission systems utilize coherent detection techniques, enabling the transmission and reception of data with high data rates and minimal signal quality degradation over long distances. Coherent detection techniques also enhance the recovery of weak optical signals, resulting in improved signal quality and enhanced resistance to noise and distortions. However, coherent optical systems are sensitive to in-phase/quadrature-phase (IQ) mismatch. IQ mismatch refers to the mismatch in amplitude and phase between the I and Q branches, which can introduce inter-carrier interference (ICI) and severely degrade the bit error rate (BER) performance of the systems. Therefore, accurately estimating and compensating for IQ mismatch is crucial to enhance the performance of coherent optical communications.