Infomax-ICA using Hessian-free optimization

We present HF-ICA, a second-order “Hessian-free” algorithm for Infomax-ICA. Our approach achieves asymptotically quadratic convergence while retaining the memory footprint of first-order methods. Without any hyperparameter tuning, we show better convergence properties than both other approximate Newton-type methods and finely-tuned stochastic Natural Gradient Descent on EEG and fMRI data. A portable, multi-threaded and vectorized C++ implementation is made publicly available along with MATLAB and Python interfaces.