Construction and Application of a Multi-Modal Knowledge Graph Integrated with Large Language Models in the Field of Manufacturing Processes

The manufacturing process of a product serves as a bridge connecting product design to production, and it is the cornerstone ensuring product quality in the manufacturing sector. The efficient application of knowledge in the field of manufacturing processes is a crucial means of enhancing manufacturing processes. Nevertheless, currently, knowledge in this field often exists in fragmented and multimodal forms within texts or databases, making it difficult to effectively mine and uniformly represent. The poor interconnectivity among knowledge pieces hinders the provision of high-quality knowledge support for the intelligent transformation of the manufacturing process sector. The advancements in knowledge graphs (KGs) and large language models (LLMs) within vertical domains have introduced novel research avenues for the construction and application of knowledge in the manufacturing process sector, laying a solid foundation for the realization of intelligent manufacturing. Therefore, this study addresses the problems of multimodality, irregularity, difficult representation, and lack of effective application of manufacturing-process domain knowledge. It combines the development of LLMs and their collaborative paradigm with multi-modal knowledge graphs (MMKGs), explores the construction of MMKGs in manufacturing process domains augmented by LLMs to realize the unified expression of process knowledge, and utilizes the excellent processing capability of LLMs to explore the LLM-enhanced multi-modal process knowledge. Additionally, we explore the effective application of multi-modal process knowledge enhanced by the LLM. This will provide inspiration and practical application references for related enterprises to further conduct research on the collaboration between LLM and KG in the manufacturing field.