Vision-based Learning: A Novel Machine Learning Method based on Convolutional Neural Networks and Spiking Neural Networks

The aim of this study is to autonomously control a Non-holonomic Mobile Robot using novel learning-based control approaches. A Spiking Neural Network (SNN) is modeled and developed to perform the decision making in the robot . A camera captures pictures from the environment and transfers them to the Convolutional Neural Network (CNN) to extract the image features. The extracted features are subsequently given to a Multi Layer Perceptron (MLP) neural network to perform the task of image classification. The results of the object recognition carried out by the image classifier unit are then given to the SNN for decision making and extracting the optimal policy based on the current states of the environment where the agent resides . The extracted policy is given to the agent to perform an action. The result of the action either results in a positive outcome or a negative outcome. The feedback of the policy execution (outcome) is given to the SNN as reward or punishment to optimize the policy extraction.