Benchmarking the selection of the hidden-layer weights in extreme learning machines

Recent years have seen a growing interest in neural networks whose hidden-layer weights are randomly selected, such as Extreme Learning Machines (ELMs). These models are motivated by their ease of development, high computational learning speed and relatively good results. Alternatively, constructive models that select the hidden-layer weights as a subset of the data have shown superior performance than random-based ones in some cases. In this work, we present a comparison between original ELMs (i.e., ELMs where the hidden-layer weights are selected randomly, that we will call ELM-Random) and a modified version of ELMs where the hidden-layer weights are a subset of the input data (that we will call ELM-Input). We will focus our comparison on the behavior of both strategies for different sizes of the training set and different network sizes. The results on several benchmark data sets for classification problems show that ELM-Input has superior performance than ELM-Random in some cases and similar in the rest. In some cases, this general trend is observed for all sizes of the training set and all network sizes. In other cases, it is mostly observed when the size of the training set is small. Therefore, the strategy of selecting the hidden-layer weights among the data can be considered as a good alternative or complement to the standard random selection for ELMs.