The Spiral Discovery Network as an Evolutionary Model for Gradient-Free Non-Convex Optimization

The Spiral Discovery Method (SDM) was originally designed as a cognitive artifact to help manage the complexities of manually tuning parametric models in high-dimensional parametric spaces. Recently, several modifications and enhancements were proposed to SDM with the goal of making it suitable for tasks requiring automated non-convex optimization besides manual parameter configuration. The key challenge behind such enhancements - collectively referred to as Spiral Discovery Network models (SDNs) based on their network-based formulation - is how to replace human intuition in maintaining the adaptivity of the search process. In this paper, recent advances behind SDN models are summarized, and their theory is further developed with the goal of making them useful for the optimization of multi-level, hierarchical architectures. Results from experiments directed at optimizing convolutional networks on the MNIST dataset are presented in order to highlight the strengths and weaknesses of the approach.