Generating Long-Horizon Task Actions by Leveraging Predictions of Environmental States

To expand the usage of robots in daily life, it is crucial for robots to learn compounded and long-horizon tasks. To learn long-horizon tasks, the tasks are generally segmented into shorter subtasks. The segmentation of the task can be done autonomously or manually, and is often based on the environment changes such as movement of object positions. The environmental state of each subtask contains valuable information that may be used as an auxiliary input for the robot learning model. However, leveraging environmental states for robot learning has the following two difficulties: maintaining and remembering environmental states that are not observable, and predicting accurate environmental states. To address these difficulties, we propose a method that predicts environmental states and confidence of prediction to leverage environmental states. The proposed method improves learning performance in long-horizon tasks by maintaining the long-term memory of the environmental states through its prediction and considering the accuracy of the environmental states. In this study, we evaluate our method by executing a long-horizon manipulation task that requires memory of past environmental states to pick-and-place multiple cubes into appropriate boxes. The six-degree-of-freedom robot was able to correctly remember the past behavior and placed the cube into the box appropriately.