Dynamic Feature Weighting Layer in Artificial Neural Networks

In this study, a fully connected pre-layer that dynamically assigns weights to input features is developed to improve classification performance of artificial neural networks. The Hadamard product of the input vector and the output of the pre-layer is transferred to the following hidden layer so as to preserve the feature subspace. In this respect, a two-stage training mechanism is devised to learn pre-layer parameters that will yield feature representations enabling the network to make more accurate predictions. Applying the proposed approach on a sentiment detection task for product reviews compiled in two different languages, its positive contribution to the classification performance is empirically shown, and furthermore, the conception that the customized layer may be used as an independent feature weighting scheme is supported by experimental results obtained with several other fundamental classification models.