Addressing the Impact of Message-Passing Topology in GNN RL

Graph Neural Network Reinforcement Learning (GNN RL) has been widely used in robotic control applications. In designing GNN RL for robots, while kinematic topology has been traditionally used as the message-passing topology, the effect of different structural designs of message-passing topology on acquired actions has not well been studied. In this paper, we evaluated those effects on a screw task by a D'Claw ROBEL hand using three types of message-passing topologies: series, parallel and series+parallel topology. As a result, we have found that parallel topology promotes the acquisition of more similar actions in a cooperative way due to homogeneous message passings to each finger, and also series topology promotes more diverse actions in a separate way due to heterogeneous message passings to each finger. Moreover, series+parallel topology acquire diverse and cooperative actions simultaneously because it generates both homegeneous and heterogeneous messages for each finger. In future work, we plan to apply this technique to multi-task RL problems.