An Academic Information Collection System based on Large Language Model

To effectively assess a scholar's capabilities, it is essential to begin by gathering their basic information, forming the foundational step for subsequent analysis. This entails sourcing pertinent details such as the scholar's affiliation, title, research focus, and published papers from online sources. We curated a comprehensive dataset comprising the homepages of numerous scholars and develop a system to collect information. Our methodology involved harnessing search engines and webpage classifiers to locate scholars' homepages across the internet. Once identified, these pages underwent meticulous parsing to extract academic information. Recent advancements in Large Language Models (LLMs) have demonstrated remarkable proficiency across various domains, particularly in fundamental Natural Language Processing (NLP) tasks. By leveraging LLM capabilities, we conducted text extraction from webpage content, resulting in the successful compilation of academic data.