Linear Models with Many Cores and CPUs: A Stochastic Atomic Update Scheme

Linear models are fast to train, apply, and still state of the art for sparse and high dimensional problems. Their computational efficiency makes them difficult to parallelize, with the standard multi-core approaches often diverging after more than 8 cores are added. We propose a Stochastic Atomic Update Scheme (SAUS) for training linear models on many core machines. It is simple to implement, reduces the number of divergent cases, and obtains greater speedups by being able to effectively use an 80-core server.