Survey of scaling platforms for Deep Neural Networks

Deep Neural Networks have become a state of the art approach in perception processing like speech recognition, image processing and natural language processing. Many state of the art benchmarks for these algorithms are using deep learning techniques. The deep neural networks in today's applications need to process very large amount of data. Different approaches have been proposed to solve scaling these algorithms. Few approach look for providing a solution over existing big data processing platform which usually runs over a large scale commodity cpu cluster. As training deep learning workload require many small computations to be done and large communication to pass the data between layers, General Purpose GPUs seems to the best platforms to train these networks. Different approaches have been proposed to scale processing on cluster of GPU servers. We have summarized various approaches used in this regard. The human brain is very good in processing perception and takes very little space and energy as compared to today's computing platforms. Neuromorphic hardware has been developed by different research groups to provide a brain like computing platform. We will look into IBM TrueNorth system in detail in this regard. Quantum computing gives another way to look in to this problem. Quantum computer can solve many complex problems as compared to classical computer. Though this field is still very nascent, people have suggested various ways to train neural network using quantum computers We will look in to recent quantum computers developed by different organizations like D-Wave and IBM. We will also look into state of the art proposed approaches to run deep neural network algorithms using quantum computer.