SECoT: Security-Oriented Chain-of-Thought Make Large Language Models Safer

The problem of harmful content generation in LLMs arises from the varied and intricate nature of their training data. Although filtering mechanisms are employed during pre-training, completely eliminating the impact of harmful content remains challenging. Additionally, the open-ended generation nature of the model during the inference phase further complicates content control. Therefore, how to effectively constrain the model's output while preserving its generative capabilities has become a critical topic in current AI safety research. Emerging research has revealed that the Chain of Thought (CoT) prompting strategy plays a pivotal role in augmenting the problem-solving abilities of large language models (LLMs). By guiding these models to generate intermediate reasoning steps, CoT facilitates a more structured approach to addressing intricate tasks, particularly in fields like mathematics and programming. This advanced reasoning ability provides new pathways for enhancing the safety of LLMs. This research introduces a Security-Oriented Chain of Thought (SECoT) mechanism. The SECoT framework consists of three parts: analyzing the safety of the problem, reflecting on the safety strategies involved, and formulating strategies to answer the problem, ultimately providing a final safe answer based on the above analysis, safety strategies, and response strategies. The SECoT process ensures that the response is not only well-considered but also safe. Building on this, a high-quality SECoT dataset (each entry includes five components: question, safety analysis, safety strategy, response principle, and safe answer) is further constructed and used as fine-tuning data to fine-tune LLMs in an end-to-end manner. Experimental results show that the SECoT framework enhances the safety of content generated by LLMs during the inference phase. The intermediate steps of the SECoT framework help LLMs learn how to reason about safe responses, reducing the harmful output rate of the fine-tuned model to 0% and lowering the average harmfulness score from 4.75 to 2.01 (on a 5-point scale, where higher scores indicate greater harmfulness).