Comprehensive Comparative Study of CPU, GPU, and TPU Performance for Neural Network Architectures Across Diverse Datasets

The rapid growth of deep learning (DL) applications, impacted by the increasing availability of data and computational demands, creates the necessity for efficient hardware solutions for training neural networks of varying complexities. This study conducts a comprehensive comparative analysis of CPU, GPU, and TPU performances across three neural network architectures: fully connected networks (FC), convolutional neural networks (CNN), and recurrent neural networks (RNN). Our experiments, conducted across six diverse datasets, examine key metrics such as training time, memory utilization, power efficiency, and throughput, highlighting the strengths and limitations of each hardware type across various neural network models and datasets. By analyzing results across hardware types, we provide insights into the practical advantages of each processor for different DL scenarios, which in turn assists developers and practitioners in selecting the most suitable hardware for their model requirements. Furthermore, our findings suggest potential areas for future hardware and software optimizations to further enhance DL performance.