Experimental Comparison between MLP and FCCN in Convolutional Neural Networks

The fully connected cascade network (FCCN) is an emerging neural network architecture which is known to be compact, having only one structural parameter which makes it convenient to use. FCCN, just like MLP, can be used in convolutional neural networks (CNN) as the final fully connected component. However, there has not been report of the use of FCCN in CNNs. In this paper we describe two CNNs with identical structure except for the fully connected component, one of which using MLP which is called CNN-MLP and the other using FCCN which is called CNN-FCCN. These two CNNs are applied to the famous MNIST digit recognition problem and their performances are compared. We used various sizes and two loss functions (MSE and cross-entropy). The performance data we obtained suggest that the two machines offer comparable performances, whereas for MSE loss, FCCN is slightly better, and for cross-entropy loss, MLP is slightly better.