Effective Tensor Sketching via Sparsification

In this article, we investigate effective sketching schemes via sparsification for high dimensional multilinear arrays or tensors. More specifically, we propose a novel tensor sparsification algorithm that retains a subset of the entries of a tensor in a judicious way, and prove that it can attain a given level of approximation accuracy in terms of tensor spectral norm with a much smaller sample complexity when compared with existing approaches. In particular, we show that for a
k
th order
$ {d}\times \cdots \times {d}$
cubic tensor of
stable rank

$ {r}_{ {s}}$
, the sample size requirement for achieving a relative error
$\varepsilon $
is, up to a logarithmic factor, of the order
$ {r}_{ {s}}^{1/2} {d}^{ {k}/2} /\varepsilon $
when
$\varepsilon $
is relatively large, and
$ {r}_{ {s}} {d} /\varepsilon ^{2}$
and essentially optimal when
$\varepsilon $
is sufficiently small. It is especially noteworthy that the sample size requirement for achieving a high accuracy is of an order independent of
k
. To further demonstrate the utility of our techniques, we also study how higher order singular value decomposition (HOSVD) of large tensors can be efficiently approximated via sparsification.