Contents TENSOR decomposition, also called tensor factorization, is useful for representing and analyzing multi-dimensional data. Tensor decompositions have been applied in signal processing applications (speech, acoustics, communications, radar, biomedicine), machine learning (clustering, dimensionality reduction, latent factor models, subspace learning), and well beyond. These tools aid in learning a variety of models, including community models, probabilistic context-free-grammars, Gaussian mixture model, and two-layer neural networks.
Although considerable research has been carried out in this area, there are many challenges still outstanding that need to be explored and addressed; some examples being tensor deflation, massive tensor decompositions, and high computational cost of algorithms. The multi-dimensional nature of the signals and even “bigger” data, particularly in next-generation advanced information and communication technology systems, provide good opportunities to exploit tensor-based models and tensor networks, with the aim of meeting the strong requirements on system flexibility, convergence, and efficiency.
This special issue (SI) covers a broad array of research on tensor methods, tensor decompositions for signal processing and machine learning, and their applications in wireless communications, radar, biomedicine, and advanced video systems, to name a few.
The issue begins with an overview article by Chen et al., which provides a broad overview of tensor analysis in wireless communications and multiple-input multiple-output (MIMO) radar. More specifically, this paper covers topics including basic tensor operations, common tensor decompositions via canonical polyadic and Tucker factorization models, wireless communications applications ranging from blind symbol recovery to channel parameter estimation, and transmit beamspace design and target parameter estimation in MIMO radar.
Sedighin et al. devise a tensor ring (TR) decomposition algorithm, especially for tensors with inexact tensor train or TR structures, such that the TR ranks are adaptively adjusted to achieve near-optimal compression. Numerical results demonstrate that the proposed approach involves a much lower storage cost than existing TR decomposition algorithms.
Rontogiannis et al. develop some interesting new results on block-term decomposition (BTD) tensor model. BTD model is especially useful for representing data that are composed of blocks of rank higher than one. A new practically useful result by the authors here is the development of a method of BTD model selection and computation. It imposes column sparsity jointly on the factors and in a hierarchical manner and estimates the ranks as the numbers of factor columns of non-negligible magnitude. The details of the algorithm development also have some interesting turns off that can be checked in the paper.
Aidini et al. propose a tensor decomposition learning method for high dimensional signal compression, which learns an appropriate basis for each dimension from a set of training samples via constrained optimization. An end-to-end compression algorithm is also presented that includes quantization and encoding, facilitating its direct application in real-world problems.
An inexact generalized Gauss-Newton (GNN) method is developed by Vandecappelle et al. to fit the canonical polyadic decomposition of a tensor for non-least-squares entry-wise cost functions. By exploiting the multilinear structure of the Hessian approximation, considerable speedup is achieved, and the inexact GNN algorithm scales much better than the exact GGN and other conventional methods without compromising fast convergence.
Schenker et al. introduce a general algorithmic framework to solve constrained linearly coupled matrix-tensor factorization problems by utilizing alternating optimization and alternating direction method of multipliers. In particular, a large variety of constraints, regularizations, coupling structures and loss functions can be handled in a plug-and-play fashion.
Wang et al. generalize the
A tensor method called PREMA is proposed by Almutairi et al. to leverage low-rank tensor factorization tools for reconstructing finer-scale data from multiple coarse views, aggregated over different subsets of dimensions. PREMA can tackle challenging scenarios, such as partially observed data and blind disaggregation without knowledge of the aggregation patterns, via a variant of PREMA called B-PREMA.
Tichavsky et al. propose novel algorithms for structured or constrained Tucker tensor decomposition and their application in block term decomposition, tensor chain modeling, classification of handwritten digits, and the compression of convolutional layers in neural networks.
Yildirim et al. introduce a dynamic generative model, the Bayesian allocation model (BAM), for modeling count data. BAM covers various probabilistic nonnegative tensor factorization (NTF) and topic models under one general framework. A novel sequential Monte Carlo algorithm is developed for marginal likelihood estimation in BAM. The developed algorithm enables unified model scoring and estimation methods for discrete variable Bayesian networks with hidden nodes, including various NTF and topic models.
Lian et al. address the fundamental problem of tensor contraction that eliminates the sharing orders among tensors and produces a compact sub-network. With the objective of minimizing storage and computational costs while searching the optimal contraction sequence, a new data structure efficient for accelerating the search of the optimal sequence is introduced, and then is combined with a new pruning method for reducing the search space, and multi-thread optimization to further improve the execution performance.
Chachlakis et al. propose a novel framework for streaming and dynamic Tucker tensor decomposition, based on maximum L1-norm projection. The proposed method has the ability to both adapt to data subspace changes and detect/reject outliers in an online fashion. The outlier-robustness, computational efficiency, and adaptivity of the proposed method are demonstrated on real-world datasets.
Sun et al. devise two coupled or joint tensor decomposition algorithms for deep convolutional neural network compression by exploring the joint information implied in the network structure. Numerical results demonstrate that the proposed approach can achieve better compression results of deep neural networks under similar performance comparing to existing tensor decomposition based methods.
Kolbeinsson et al. present tensor dropout, a randomization technique that can be applied to tensor factorizations, such as those parametrizing tensor layers in neural networks. When applied to those tensor layers, tensor dropout significantly improves the robustness and generalization of the network. Empirical experiments on large scale MRI and image-based prediction show that tensor dropout leads to superior performance and improved robustness against random and adversarial noise.
Tsai et al. devise a tensor-based reinforcement learning to solve the network routing and flow scheduling problem. Applying the Tucker decomposition into Q-learning not only can improve the learning quality of each episode but also lower the sample complexity. The simulation results demonstrate that the proposed approach can achieve more stable results with less convergence time than traditional routing approach and conventional reinforcement learning techniques and comparable results to deep Q-learning.
Ding et al. suggest modeling spectral images as tensors following the block-term decomposition framework with multilinear rank-(
Weitzner and Giryes present a separable approach to blind deconvolution and demixing via convex optimization. The proposed formulation allows separation into smaller optimization problems, which significantly improves complexity. Recovery guarantees are provided which comply with those of the original non-separable problem, and performance is demonstrated under several normalization constraints.
From the perspective of exploiting the fifth-order tensor representation of light field image (LFI) and its distortion characteristics included in the spatial and angular dimensions of LFI, Pan et al. propose a method of combining tensor slice and singular value for blind light field image quality assessment, which matches well with human subjective opinions.
Hoang et al. provide a new tensor augment method to cast a low order tensor of color images or videos to a higher order tensor such that the latter captures all correlations and entanglements between the entries of the former. Completing such higher tensor helps to achieve high recovery rates even for high-definition images and videos with 95% missing pixels.
Borsoi et al. developed a hyperspectral and multispectral image fusion method based on the Tucker and block term tensor decompositions to account for inter-image variability. Theoretical guarantees for the recovery of the high-resolution image were provided, and numerical simulations showed an improved experimental performance of the method.
Wang et al. propose a Tensor low-rank (TLR) constraint and an
Hu et al. develop a lightweight 2-D convolutional long short-term memory (ConvLSTM2D) unit by utilizing tensor-train decomposition, and propose a lightweight but lossless tensor attention-driven ConvLSTM neural network, where it is the first attempt of constructing a tensor attention structure by combining tensor representation of hyperspectral data for classification. Experimental results illustrate that the proposed model can effectively reduce trainable parameters and achieve higher classification accuracies than other competitive algorithms.
Taguchi et al. applied a tensor decomposition (TD)-based unsupervised feature extraction approach to a gene expression profile dataset that is regarded as a suitable model of human coronavirus infection. The tensor method selected 134 altered genes, which can represent the coronavirus infectious process. Drug compounds were then selected targeting the expression of these genes based on a public domain database. The identified drug compounds were mainly related to known antiviral drugs, several of which were also previously identified via in silico experiments for treating COVID-19.
Zhang et al. propose a tensor-train deep neural network (TT-DNN)-based channel estimator. The advantage of the TT-DNN-based approach is that it allows to tackle challenges of time-varying channel estimation in MIMO systems. The distributed block-by-block and antenna-by-antenna implementations of the TT-DNN-based channel estimation are discussed with the objective of improving the flexibility and computational efficiency.
Chou et al. propose a data-driven position-aided approach to reduce the training overhead in MIMO systems. A signal tensor is constructed by collecting beam-training measurements on a subset of positions and beams. Exploiting prior knowledge of the spatial smoothness and the low-rank property of MIMO channels, a noisy tensor completion algorithm is proposed for interpolating/predicting the received power across the whole coverage area. The value of the paper comes also from the fact that the proposed tensor completion algorithm is validated by actual experiments based on the Quadriga channel simulator at 60 GHz millimeter-wave channels.
de Araujo et al. address the use of intelligent reflecting surfaces in 5G and 6G wireless communication systems. The reflected signals add coherently at the receiver or destructively to reduce co-channel interference. To enable receiver design, accurate channel estimation techniques are required, where a tensor modeling approach based on supervised (pilot-assisted) methods is developed, and two specific methods that exploit a structured time-domain pattern of pilots and reflecting surface phase shifts are proposed.
Sokal et al. propose also a tensor modeling and decomposition based technique for joint channel, data, and phase-noise estimation in MIMO-OFDM systems. First it is established that the received signal in such system at the pilot subcarriers can be modeled as a third-order PARAFAC tensor. then using this tensor signal model two algorithms for channel and phase-noise estimation at the pilot subcarriers are proposed. A zero-forcing received based on the PARATuck tensor structure of the received signal at the data subcarriers is than developed for data estimation.
Zhang et al. address also the channel estimation problem in MIMO-OFDM systems in the currently importation hybrid millimeter wave context. One of the major problem then is the development of gridless channel estimation methods and it is addressed in this paper by proposing an algorithm based on the 3D tensor-ESPRIT in DFT beamspace framework.
Cheng and Shi also address the channel estimation problem, but in the context of multi-user 3D massive MIMO communications. Indeed, the channel estimation for 3D massive MIMO communications has been improved very significantly by exploring the angular channel model and tensor decomposition. However, the specific problem with channel estimation in multi-user systems is the coupling among multiple users channels. To overcome this problem, the authors develop a new tensor-based channel model and a channel estimation algorithm based on block coordinate descent is proposed, and the approach is coined as overfitting avoidance.
Chang et al. develop the sparse signal recovery and denoising algorithm on tensor systems via modifying the prior distribution of the interesting parameters in the traditional sparse Bayesian learning. This modification not only reduces the complexity of the Kronecker-like linear inverse problem but also enhances the accuracy of the estimation in some applications, such as mm-wave MIMO channel estimation with the uniform planar array.
We conclude this editorial with the hope that this SI is informative and spurs future developments and advancements of tensor decompositions within the signal processing community and beyond. Finally, we would like to express our thanks to all the authors for their contributions to this SI and the reviewers for their careful assessments and timely responses. We are very grateful to the Editor-in-Chief, Dr. Lina Karam, and the Publications Administrator, Mikaela Langdon, for their invaluable advice and prompt assistance throughout the process of putting this SI together.
Hongyang Chen, Lead Guest Editor Research Center for Intelligent Network Zhejiang Lab, China e-mail: hongyang@zhejianglab.com Sergiy A. Vorobyov, Guest Editor Department of Signal Processing and Acoustics Aalto University, Finland e-mail: sergiy.vorobyov@aalto.fi Hing Cheung So, Guest Editor Department of Electrical Engineering City University of Hong Kong, Hong Kong e-mail: hcso@ee.cityu.edu.hk Fauzia Ahmad, Guest Editor Department of Electrical and Computer Engineering Temple University, USA e-mail: fauzia.ahmad@temple.edu Fatih Porikli, Guest Editor Reseach School of Engineering Australian National University, Australia e-mail: fatih.porikli@anu.edu.au