Automatic Prompt Generation Based on Combinatorial Relationships in Reasoning Problems

The effectiveness of Large Language Models (LLMs) in tasks involving reasoning is significantly influenced by the structure and formulation of the prompts, contemporary research in prompt engineering aims to help LLMs better understand the paradigms of reasoning question (e.g., CoT). However, these efforts have either struggled to effectively incorporate external knowledge into single prompt or integrating entire corpus information, often fails to significantly enhance the reasoning capabilities of LLMs. This paper introduces a novel prompting method that incorporates implicit hints that represent logical combinatorial relationships between known conditions in reasoning problems, guiding LLMs to think correctly in the initial steps of reasoning for such problems. Extensive and comprehensive experiment results on four different reasoning problem datasets indicate that our proposed method improved accuracy while maintaining efficiency.