InvisiCode: Boosting Intra-Frame Screen-Camera Communication by Breaking Through Noise Limitations

Screen-camera communication enables the seamless integration of encoded auxiliary information from the digital world into the physical domain—allowing users to obtain detailed information about an object of interest, such as a poster, simply by capturing a photo with a smartphone. Traditional screencamera communication methods, such as barcodes, occupy visual space and degrade aesthetics. While inter-frame encoding methods address these limitations, they are restricted to video streams or active screen displays. To enable content-preserving intraframe screen-camera communication, we propose InvisiCode, a noise-aware method for imperceptible, robust, and high-capacity encoding. We first quantitatively analyze screen-camera noise and identify predictable patterns in mid-high frequency Discrete Cosine Transform (DCT) coefficients, enabling mathematically bounded, noise-aware encoding. Based on this insight, we design an adaptive encoding algorithm that distributes data across multiple coefficients, balancing imperceptibility and resilience to noise. To ensure accurate decoding, we enhance $\mathrm{U}^{2}$-Net with Edge-Constraint Loss to improve boundary detection and precisely locate the encoded region in captured images. Experimental results demonstrate that InvisiCode is reliable and adaptable across various screen and camera configurations, including smartphones, tablets, laptops, and desktop monitors. It achieves a throughput of 784 bits per frame with a Bit Error Rate (BER) of less than 0.05, significantly outperforming previous methods. User studies confirm that the system introduces imperceptible distortion. Our code and demo are available at https://github.com/haikuoY/InvisiCode.