Construction of a Quality Evaluation Index System for Response Texts in Medical AI Agents

This study tackles the challenge of evaluating medical AI agents' response text quality by constructing a hierarchical system with 3 primary, 6 secondary, and 19 tertiary indicators based on product quality signaling and information quality theories. Its reliability and validity were verified through a questionnaire survey, and an improved CRITIC-based weighting method was proposed to enhance the objectivity of weight assignment. This framework provides a theoretical and practical basis for performance evaluation, model optimization, and trust enhancement in medical AI agents, contributing to the sustainable development of the "Internet + Healthcare" ecosystem. Future work may explore index adaptation across medical subdomains and incorporate dynamic weight adjustment mechanisms.