Tool Path Optimization for Complex Cavity Milling Based on Reinforcement Learning Approach

In the machining of parts, tool paths for complex cavity milling often have different generation options, as opposed to simple machining features. The different tool path generation options influence the machining time and cost of the part during the machining process. Decision makers prefer tool path solutions that have fewer blanking lengths, which means that the machining process is more efficient. Therefore, in order to reduce costs and increase efficiency, it is necessary to carefully design the tool path generation for the features to be machined on the part, especially for complex cavity milling features. However, solutions to the problem of optimal design of tool paths for complex cavity milling features have not been well developed in current research work. In this paper, we present a systematic solution for complex cavity milling tool path generation based on reinforcement learning. First, a grid converter is executed for converting the 3D geometry of the cavity milling feature into a matrix of planar grid points recognisable by the program, set according to the cutting parameters. Afterwards, the tool path generation process is refined and modelled as a Markov decision process. Ultimately, a tool path generation solution combining the A* algorithm with the Q-learning algorithm is executed. The agent iterates through trial and error to construct an optimal tool path for a given cavity milling task. Three case experiments demonstrate the feasibility of the proposed approach. The superiority of the reinforcement learning-based approach in terms of solution speed and solution quality is further demonstrated by comparing the proposed approach with the evolutionary computational techniques currently popular in research for solving tool path optimisation design problems.