SPAFormer: Sequential 3D Part Assembly with Transformers

We introduce SPAFormer, an innovative model designed to overcome the combinatorial explosion challenge in the 3D Part Assembly (3D-PA) task. This task requires accu-rate prediction of each part's poses in sequential steps. As the number of parts increases, the possible assembly com-binations increase exponentially, leading to a combinato-rial explosion that severely hinders the efficacy of 3D-PA. SPAFormer addresses this problem by leveraging weak con-straints from assembly sequences, effectively reducing the solution space's complexity. Since the sequence of parts conveys construction rules similar to sentences structured through words, our model explores both parallel and au-toregressive generation. We further strengthen SPAFormer through knowledge enhancement strategies that utilize the attributes of parts and their sequence information, enabling it to capture the inherent assembly pattern and relationships among sequentially ordered parts. We also construct a more challenging benchmark named PartNet-Assembly cov-ering 21 varied categories to more comprehensively val-idate the effectiveness of SPAFormer. Extensive experi-ments demonstrate the superior generalization capabilities of SPAFormer, particularly with multi-tasking and in sce-narios requiring long-horizon assembly. Code is available at https://github.com/xuboshen/SPAFormer.