Enhancing Multimodal Chain-of-Thought Reasoning with Tree-Searched Self-Training

Despite impressive performance on general visual benchmarks, Multimodal Large Language Models (MLLMs) still struggle with generating consistent and accurate reasoning processes for complex visual reasoning tasks. The limited availability of multimodal reasoning datasets further complicates fine-tuning efforts to improve reasoning capabilities. To address these challenges, we introduce Tree-Searched Self-Training (TSST), a novel framework that enhances multimodal reasoning through self-improvement without relying on extensive manual annotations. TSST introduces a hierarchical tree-search mechanism that combines stepwise rationales generation with value-guided path selection, enabling the model to explore and identify high-quality reasoning trajectories autonomously. Training on these self-generated data significantly improves the model’s reasoning ability, boosting consistency and accuracy. Extensive experiments on ScienceQA and VCR benchmarks show that TSST outperforms existing fine-tuning baselines by 6.0% in accuracy, demonstrating its effectiveness in enhancing multimodal reasoning capabilities. The code is publicly available at: https://github.com/laaambs/tsst.