Babouk: Focused Web Crawling for Corpus Compilation and Automatic Terminology Extraction

The use of the World Wide Web as a free source for large linguistic resources is a well-established idea. Such resources are keystones to domains such as lexicon-based categorization, information retrieval, machine translation and information extraction. In this paper, we present an industrial focused web crawler for the automatic compilation of specialized corpora from the web. This application, created within the framework of the TTC project, is used daily by several linguists to bootstrap large thematic corpora which are then used to automatically generate bilingual terminologies.