Extracting fuzzy symbolic representation from artificial neural networks

The paper presents FUZZYTREPAN, a pedagogical approach to the problem of extracting comprehensible symbolic knowledge from trained artificial neural networks. This approach extends the previously proposed TREPAN method in two ways: it uses fuzzy representation in its knowledge extraction process (by means of fuzzy decision trees), and it uses additional heuristics in its process of generating artificial data. The paper describes the proposed approach in detail, and it presents its empirical evaluation on popular machine learning benchmarks.