Two-Input Power-Activation Neural Network Weights-Direct-Determination and Structure Optimized by Particle Swarm optimization

Artificial network, with back-propagation (BP) training algorithms, has shown inherent weaknesses in determining weights and optimal network structure. In order to overcome those weaknesses, in this paper, a two-input single-output (TISO) power-activation feed-forward neural network model is investigated and constructed, based on theory of multivariate approximation and power series expansion. Then, the weights-direct-determination (WDD) method for our feed-forward network is studied and proposed. Further, in order to optimize the structure of network, particle swarm optimization (PSO) is applied to search for global optimal solution to the use of neurons in hidden layer. Then, these lead us finally to propose a weights-structure-determination (WSD) methodology by combining WDD and PSO novelly. Finally, computer numerical experiments based on various objective functions substantiate the superiority of our network in terms of approximation and denoising. Further comparison experiments with stochastic gradient-descent (SGD) and previous work substantiate the efficacy of the WSD methodology proposed.