Thinktank: Leveraging LLM Reasoning for Advanced Task Execution in CI/CD

Thinktank is a task execution framework that harnesses the reasoning capabilities of large language models (LLMs). These models demonstrate an ability to provide intermediate steps essential for approaching problem solutions. Thinktank capitalizes on this by employing an LLM, such as GPT-4, to iteratively solve given objectives. Initially, Thinktank processes each objective into actionable tasks that can be executed immediately using the available information. We employ a technique called ReAct prompting (https://arxiv.org/abs/2210.03629) to leverage the LLM’s reasoning abilities, guiding it to select appropriate Agents. Our method deviates from the original paper’s proposal; recognizing that the vast number of available functions could overwhelm the decision-making of statistical language models, we restrict the model’s function choices by automatically clustering Agents into no more than ten capabilities. These capabilities are dynamically recalculated whenever new Agents are added to the system.