SecureQNN: Shielding the Intellectual Property of QNNs in TinyML Systems

Building accurate Machine Learning (ML) models requires substantial expertise and large-scale datasets typically only available in big data companies. These companies have been selling their models as Machine Learning as a Service. However, concerns about data privacy and the appliance of ML in mission-critical scenarios are forcing ML computation to move from the cloud to the deep edge, near sensor data. If edge ML increases user data privacy and makes decision latency predictable, deploying proprietary ML models on untrusted edge devices may harm the intellectual property of Service Providers. A natural response to this issue comes from Trusted Execution Environments (TEEs), which provide hardware-based security. However, adapting ML computation to the constraints of TEEs remains an open challenge. In this article, we propose SecureQNN, a framework that leverages state-of-the-art TEE technology (TrustZone-M) available in Arm Cortex-M microcontrollers to increase the protection of Quantized Neural Networks (QNNs) against unauthorized replication. SecureQNN evaluates which layers lessen the effort of an attacker when building a surrogate QNN with the same accuracy. The layers making the attacker spend less training epochs than the owner training from scratch are stored and executed in the secure-world of the TEE. Experiments demonstrate that SecureQNN undermines the cost-benefit of unauthorized QNN replication by isolating 51%–65% of the model size while incurring a worst-case decision latency overhead of only 0.06%. Although SecureQNN has broader applicability, the negligible impact on decision latency and its deterministic behavior highlight the suitability of SecureQNN for mission-critical and real-time applications.