AugOracle: In-Capability Raw Input Validation for Deep Learning Models in Deployment

Software engineering community has developed effective techniques to detect the failures of software systems with deep learning model components from out-of-distribution inputs. However, failures from in-distribution inputs are underexplored. In this paper, we formulate the problem of in-capability raw input validation regarding the in-distribution failures and propose AugOracle, an effective yet efficient input validation technique, to address this problem. AugOracle differs from previous works in two key aspects: (1) efficiency - it provides robustness requirement rules for detection rather than heavy training-based methods; (2) scalability - it can scale to larger deep learning models in constant overhead. We evaluate AugOracle on 27 configurations to show its effectiveness and efficiency and open-source the implementation.