Forced and Natural Creative-Prototype Learning for Interpreting Multi-Layered Neural Networks

This paper aims to demonstrate the existence of prototype networks and their learning process. Prototype learning aims to acquire the simplest form or configuration within the given network resources. This prototype learning is assumed to be naturally integrated into conventional learning, referred to as “natural learning”. Additionally, prototype learning in this natural learning is augmented by separately employing prototype learning, which can be termed “forced learning”. The existence of prototype networks is not easily identified, as they are sometimes weakly activated. To explicitly identify prototype networks, we introduce potentiality and corresponding ratio potentiality. They are analogous to entropy and divergence in conventional information theoretic methods but are developed for interpretation, aiming to clarify more detailed characteristics of learning. The method was preliminarily applied to the bankruptcy dataset, which was small, but it has been challenging to improve and interpret the final results. The results confirmed that the new method could elucidate the existence of prototype networks at the beginning of learning in the natural learning paradigm. This identification of prototype networks was reinforced by employing the forced method. Finally, the new method demonstrated the possibility of discovering new relationships between inputs and outputs beyond linear correlation coefficients.