Test Generation from Use Case Specifications for IoT Systems: Custom, LLM-Based, and Hybrid Approaches

IoT systems are increasingly developed and deployed across various domains, where End-to-End (E2E) testing is critical to ensure reliability and expected behavior. However, generating comprehensive tests remains challenging due to the heterogeneity, distributed nature, and unique characteristics of IoT systems, which limit the effectiveness of generic test generation approaches. Recent studies demonstrated the effectiveness of Large Language Models (LLM) for test generation in traditional software systems. Building on this foundation, this study explores and evaluates four distinct approaches for generating E2E tests from use case specifications (UCSs) tailored to IoT systems. These include (1) a custom, (2) a single-stage LLM, (3) a multi-stage LLM, and (4) a hybrid approach combining custom and LLM capabilities. We evaluated these approaches on an IoT system, focusing on correctness and scenario coverage criteria. Experimental results indicate that all approaches perform well, with notable variations in specific aspects of test generation. The custom and hybrid approaches are more reliable in producing correctly structured and complete tests, with the hybrid approach slightly outperforming others. This study is a work in progress, requiring further investigation to fully realize its potential.