Adaptive Reward Computation in Reinforcement Learning-Based Continuous Integration Testing

Reinforcement learning (RL) has been applied to prioritizing test cases in Continuous Integration (CI) testing, where the reward plays a crucial role. It has been demonstrated that historical information-based reward function can improve the effectiveness of the test case prioritization (TCP). However, the inherent character of frequent iterations in CI can produce a considerable accumulation of historical information, which may decrease TCP efficiency and result in slow feedback. In this paper, the partial historical information is considered in the reward computation, where sliding window techniques are adopted to capture the possible efficient information. Firstly, the fixed-size sliding window is introduced to set a fixed length of recent historical information for each CI test. Then dynamic sliding window techniques are proposed, where the window size is continuously adaptive to each CI testing. Two methods are proposed, the test suite-based dynamic sliding window and the individual test case-based dynamic sliding window. The empirical studies are conducted on fourteen industrial-level programs, and the results reveal that under limited time, the sliding window-based reward function can effectively improve the TCP effect, where the NAPFD (Normalized Average Percentage of Faults Detected) and Recall of the dynamic sliding windows are better than that of the fixed-size sliding window. In particular, the individual test case-based dynamic sliding window approach can rank 74.18% failed test cases in the top 50% of the sorting sequence, with 1.35% improvement of NAPFD and 6.66 positions increased in TTF (Test to Fail).