An Investigation of the Probability Distribution Patterns of Parameters in the Well-trained Vision Transformer Model

To the best of our knowledge, this is the first time that the distribution pattern of the big-model (DINO V2) parameters has been investigated. Our findings verified the breakdown of the standard Gaussian distribution pattern in deep learning, and according to this characteristic, we provide some worth-trying advice on how to fine-tune the big-model in RS interpretation tasks.