A study of particle swarm optimization for cognitive machines

This paper presents a study of the properties of optimization algorithms for use in cognitive machines through five key measures: (i) speed of convergence, (ii) degree of exploration of the parameter space, (iii) storage and system size, (iv) adaptability, and (v) multi-scale capabilities. Based on these factors, a novel study of the trajectories of a particle in the particle swarm optimization algorithm is performed both in the time and frequency domain. The analysis shows that the trajectories of particles can be separated into a transient and a steady state periods where the transient is wide-sense stationary with long term dependancies that show the evolutionary properties of the algorithm as it converges on a solution. The steady state shows an increased degree of exploration of the parameter space that allow the algorithm to improve on the solution found over time. The results show the advantages of particle swarm optimization and inherent properties that make this optimization algorithm a suitable choice for use in cognitive machines. The information learned from this analysis can further be used to extract complexity measures to classify the behavior and control of particle swarm optimization, and make proper quick decisions on what to do next. The decision process often requires more alternatives to be considered in a short window of time than it is physically possible for a real-time system [Kins04]. Thus, in order to make good decisions without exploring all possible paths, a cognitive system requires optimization techniques that can survey the possible options, and quickly select the best option possible. The paper reviews the requirements for an ideal optimization technique for use in cognitive systems and proposes the particle swarm optimization algorithm as one technique that is designed to satisfy these requirements. In order to show the properties of PSO, a novel trajectory, time, and frequency domain analyses of single particles along individual dimensions are presented.