The Least Square QR Method Improves Extreme Learning Machine

The Extreme Learning Machine (ELM) is a fast and straightforward learning algorithm designed for single hidden layer feed-forward neural networks (SLFNs). The ELM randomly assigns input weights and calculates the output weights analytically using the Moore-Penrose generalized inverse (MP) (also called Pseudo inverse). However, the MP is not preferable for ill-posed systems as it is sensitive to perturbation and may produce inaccurate solutions. Furthermore, it is computationally expensive as it involves matrix inversion. Regularized ELM (RELM) was proposed to address the limitations of the MP. In RELM, the regularization parameter is the primary factor that controls the classifier's performance. Though the RELM is stable and produces accurate results, finding a n a ppropriate regularization parameter value is tedious and increases the computation. This work proposes a variant of ELM named LSQR-ELM algorithm, which balances the ELM and RELM by maintaining stability and accuracy and avoiding the regularization entirely. The LSQR-ELM is based on the Least Squares QR (LSQR) iterative technique. Since the LSQR finds p a rtial Q R rather than full QR, it reduces unnecessary computation. To verify the proposed algorithm's ability, comparisons are made with several existing methods including ELM, Minimum Residual Method (MT-ELM), SVD-based SG-ELM, Optimally Pruned ELM (OP-ELM), and K-nearest neighbor (KNN). Extensive experimen-tation on various datasets demonstrates that the LSQR-ELM algorithm outperforms ELM, SG-ELM, OP-ELM, and KNN on most datasets. Moreover, it exhibits comparable performance to MT-ELM while requiring significantly f e wer computations.