CATO: A Transformer Model for Augmented Temporal Decision Making with Offline Reinforcement Learning

Decision transformer (DT) mainly relies on global self-attention. It tends to overlook local features and may generate out-of-distribution actions due to cumulative errors in offline training. To address the above problems, a Convolution-Augmented Transformer with an OOD constraint (CATO) model is proposed in this paper. The core innovations of the model include a multi-level grouping feature extraction and fusion module, and a convolution branch integrated within the attention mechanism for balancing global and local features; an OOD-aware constraint module is constructed to constrain the distribution of actions and suppress the generation of out-of-distribution actions. The experimental results demonstrate that CATO achieves an average normalized score of 80.4 across nine MuJoCo tasks on the D4RL benchmark, outperforming DT by 7.6% (74.7 points). This work provides a novel framework for offline reinforcement learning and has wide application potential in complex temporal decision-making tasks.