A Large Language Models Security Performance Evaluation Method Based on Analytic Hierarchy Process and Shapley Value Fusion

This paper investigates a method for assessing the safety performance of Large Language Models (LLMs) based on the fusion of Analytic Hierarchy Process (AHP) and Shapley values. By weighted fusion of AHP subjective weights and Shapley objective weights, the final comprehensive weights are obtained and the model security score is calculated, and the model security score is used to assess the model security performance. The experimental results show that the security performance of LLMs is importantly linked to the attack success rate and toxicity detection rate, and the security performance of GPT-4 is optimal among the five LLMs set up in the experiment. This method can evaluate the security performance of LLMs more scientifically.