StegGuard: Secrets Encoder and Decoder Act as Fingerprint of Self-Supervised Pretrained Model

In this work, we propose StegGuard, a novel fingerprinting mechanism to verify the ownership of a suspect pretrained model using steganography, where the pretrained model is obtained via self-supervised learning. A critical perspective in StegGuard is that the unique characteristic of the transformation from an image to an embedding, conducted by the pretrained model, can be equivalently captured by how an encoder embeds secrets into images and how a decoder extracts them from the embeddings with tolerable error. While each independently trained pretrained model has a distinct transformation, a piracy model exhibits a transformation similar to that of the victim. Based on these observations, StegGuard learns a pair of secrets encoder and decoder as the fingerprint of the victim model. Additionally, a Frequency domain channel attention Embedding block is introduced into the encoder to adaptively embed secrets into suitable frequency bands. During verification, if the secrets embedded into the query images can be extracted with an acceptable error from the embeddings of the query images, the suspect model is determined to be piracy; otherwise, it is deemed independent. Extensive experiments demonstrate that with as few as 100 query images, StegGuard achieves high piracy detection accuracy and robustness against model stealing attacks, including model extraction, fine-tuning, pruning, embedding noising and shuffle. Compared to existing methods, StegGuard consistently achieves lower p-values for piracy models (as low as 1e-14) and higher p-values for independent models (up to 0.99), confirming its effectiveness and reliability.