A Hybrid Regularized Multilayer Perceptron for Input Noise Immunity

The immunity of multilayer perceptron (MLP) is less effective toward input noise. In this article, we have focused on the robustness of MLP with respect to input noise where noise can be additive or multiplicative. Here, we have proposed a DropConnect-based regularized MLP to reduce the coadaptation among the neurons of the hidden layer. At first, we have empirically and statistically shown that by reducing coadaptation among the hidden neurons, an MLP can achieve better noise immunity. We have also empirically shown that an MLP with input noise injection and
$l_{2}$
regularizer is an effective approach to improve its noise immunity. However, the results indicate that it does not adjust the coadaptation among the hidden neurons. Therefore, for further improvement, we have proposed a hybrid regularized MLP (HRMLP), where DropConnect is combined with the noise injection and
$l_{2}$
regularizer. In addition to input noise, we have also verified the robustness of HRMLP with respect to
$\mathbf {20\%}$
outliers in the dataset. To justify the effectiveness of the proposed HRMLP, we have compared it with MLP, MLP with noise injection, MLP with
$l_{2}$
regularizer, MLP with noise injection and
$l_{2}$
regularizer, and MLP with DropConnect along with two state-of-the-art works based on
$\mathbf {20}$
standard datasets. The experimental results for both noisy inputs and outliers confirm that the performance of HRMLP is significant compared to other methods.