Integrated Sensing and Communications with MIMO-OTFS: ISI/ICI Exploitation and Delay-Doppler Multiplexing

Orthogonal time frequency space (OTFS) is a promising alternative to orthogonal frequency-division multiplexing (OFDM) for high-mobility communications. We propose a novel multiple-input multiple-output (MIMO) integrated sensing and communication (ISAC) system based on OTFS modulation. We begin by deriving new sensing and communication signal models for the proposed MIMO-OTFS ISAC system that explicitly capture inter-symbol interference (ISI) and inter-carrier interference (ICI) effects. We then develop a generalized likelihood ratio test (GLRT) based multi-target detection and delay-Doppler-angle estimation algorithm for MIMO-OTFS radar sensing that can simultaneously mitigate and exploit ISI/ICI effects, to prevent target masking and surpass standard unambiguous detection limits in range/velocity. Moreover, considering two operational modes (discovery/track), we propose an adaptive MIMO-OTFS ISAC transmission strategy. For the discovery mode, we introduce the concept of delay-Doppler (DD) multiplexing, enabling omnidirectional probing of the environment and large virtual array at the OTFS radar receiver. For the track mode, we pursue a directional transmission approach and design an OTFS ISAC optimization algorithm in spatial and DD domains, seeking the optimal trade-off between radar signal-to-noise ratio (SNR) and achievable rate. Simulation results verify the effectiveness of the proposed sensing algorithm and reveal valuable insights into OTFS ISAC trade-offs under varying communication channel characteristics.