How to Train Your Unstable Looped Tensor Network

This paper addresses a substantial question of how to compress Deep Neural Networks with convolutional kernels modeled as looped tensor networks or Tensor Chain (TC) while it is known that such tensor network (TN) encounters severe numerical instability. We study the perturbation of this TN, provide an interpretation of instability in TC, propose novel methods to gain stability of the decomposition and keep the tensor network robust, and attain better approximation. Experimental results will confirm the superiority of the proposed methods in the compression of well-known convolutional neural networks, and TC decomposition under challenging scenarios.