On training piecewise linear networks

Piecewise-linear (PWL) neural networks are networks with piecewise-linear node functions. They have attractive features, such as speed of training and amenability to digital VISI implementation. The paper presents an algorithm for training PWL networks. The algorithm is general in that it can be used for the optimization of general PWL functions. It is based on moving from one linear region to the next. This is achieved by exploring 2N specific directions along the boundaries between the linear regions (N is the dimension), and moving along the direction that achieves a descent in the objective function. Convergence to the local minimum is proved, and simulation results confirm the computational efficiency of the algorithm.