# IEEE Transactions on Information Theory

Includes the top 50 most frequently accessed documents for this publication according to the usage statistics for the month of

• ### Compressed sensing

Publication Year: 2006, Page(s):1289 - 1306
Cited by:  Papers (12460)  |  Patents (164)
| | PDF (483 KB) | HTML

Suppose x is an unknown vector in Ropf<sup>m</sup> (a digital image or signal); we plan to measure n general linear functionals of x and then reconstruct. If x is known to be compressible by transform coding with a known transform, and we reconstruct via the nonlinear procedure defined here, the number of measurements n can be dramatically smaller than the size m. Thus, certain natural... View full abstract»

• ### Channel Polarization: A Method for Constructing Capacity-Achieving Codes for Symmetric Binary-Input Memoryless Channels

Publication Year: 2009, Page(s):3051 - 3073
Cited by:  Papers (1239)  |  Patents (26)
| | PDF (707 KB) | HTML

A method is proposed, called channel polarization, to construct code sequences that achieve the symmetric capacity I(W) of any given binary-input discrete memoryless channel (B-DMC) W. The symmetric capacity is the highest rate achievable subject to using the input letters of the channel with equal probability. Channel polarization refers to the fact that it is possible to synthesize, out of N ind... View full abstract»

• ### Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information

Publication Year: 2006, Page(s):489 - 509
Cited by:  Papers (7629)  |  Patents (141)
| | PDF (1167 KB) | HTML

This paper considers the model problem of reconstructing an object from incomplete frequency samples. Consider a discrete-time signal f/spl isin/C/sup N/ and a randomly chosen set of frequencies /spl Omega/. Is it possible to reconstruct f from the partial knowledge of its Fourier coefficients on the set /spl Omega/? A typical result of this paper is as follows. Suppose that f is a superposition o... View full abstract»

• ### Speeding Up Distributed Machine Learning Using Codes

Publication Year: 2018, Page(s):1514 - 1529
Cited by:  Papers (22)
| | PDF (1461 KB) | HTML

Codes are widely used in many engineering applications to offer robustness against noise. In large-scale systems, there are several types of noise that can affect the performance of distributed machine learning algorithms-straggler nodes, system failures, or communication bottlenecks-but there has been little interaction cutting across codes, machine learning, and distributed systems. In this pape... View full abstract»

• ### Channel Coding Rate in the Finite Blocklength Regime

Publication Year: 2010, Page(s):2307 - 2359
Cited by:  Papers (747)  |  Patents (1)
| | PDF (3873 KB) | HTML

This paper investigates the maximal channel coding rate achievable at a given blocklength and error probability. For general classes of channels new achievability and converse bounds are given, which are tighter than existing bounds for wide ranges of parameters of interest, and lead to tight approximations of the maximal achievable rate for blocklengths <i>n</i> as short as 100. It is... View full abstract»

• ### Factor graphs and the sum-product algorithm

Publication Year: 2001, Page(s):498 - 519
Cited by:  Papers (2993)  |  Patents (192)
| | PDF (458 KB) | HTML

Algorithms that must deal with complicated global functions of many variables often exploit the manner in which the given functions factor as a product of "local" functions, each of which depends on a subset of the variables. Such a factorization can be visualized with a bipartite graph that we call a factor graph, In this tutorial paper, we present a generic message-passing algorithm, the sum-pro... View full abstract»

• ### Decoding by linear programming

Publication Year: 2005, Page(s):4203 - 4215
Cited by:  Papers (3259)  |  Patents (43)
| | PDF (375 KB) | HTML

This paper considers a natural error correcting problem with real valued input/output. We wish to recover an input vector f/spl isin/R/sup n/ from corrupted measurements y=Af+e. Here, A is an m by n (coding) matrix and e is an arbitrary and unknown vector of errors. Is it possible to recover f exactly from the data y? We prove that under suitable conditions on the coding matrix A, the input f is t... View full abstract»

• ### Nearest neighbor pattern classification

Publication Year: 1967, Page(s):21 - 27
Cited by:  Papers (2056)  |  Patents (98)
| | PDF (1019 KB)

The nearest neighbor decision rule assigns to an unclassified sample point the classification of the nearest of a set of previously classified points. This rule is independent of the underlying joint distribution on the sample points and their classifications, and hence the probability of error<tex>R</tex>of such a rule must be at least as great as the Bayes probability of error<tex... View full abstract»

• ### Cooperative diversity in wireless networks: Efficient protocols and outage behavior

Publication Year: 2004, Page(s):3062 - 3080
Cited by:  Papers (8868)  |  Patents (63)
| | PDF (572 KB) | HTML

We develop and analyze low-complexity cooperative diversity protocols that combat fading induced by multipath propagation in wireless networks. The underlying techniques exploit space diversity available through cooperating terminals' relaying signals for one another. We outline several strategies employed by the cooperating radios, including fixed relaying schemes such as amplify-and-forward and ... View full abstract»

• ### List Decoding of Polar Codes

Publication Year: 2015, Page(s):2213 - 2226
Cited by:  Papers (248)
| | PDF (1015 KB) | HTML

We describe a successive-cancellation list decoder for polar codes, which is a generalization of the classic successive-cancellation decoder of Arıkan. In the proposed list decoder, L decoding paths are considered concurrently at each decoding stage, where L is an integer parameter. At the end of the decoding process, the most likely among the L paths is selected as the single codeword at the deco... View full abstract»

• ### A Mathematical Theory of Deep Convolutional Neural Networks for Feature Extraction

Publication Year: 2018, Page(s):1845 - 1866
Cited by:  Papers (8)
| | PDF (905 KB) | HTML

Deep convolutional neural networks (DCNNs) have led to breakthrough results in numerous practical machine learning tasks, such as classification of images in the ImageNet data set, control-policy-learning to play Atari games or the board game Go, and image captioning. Many of these applications first perform feature extraction and then feed the results thereof into a classifier. The mathematical a... View full abstract»

• ### Sparse Solution of Underdetermined Systems of Linear Equations by Stagewise Orthogonal Matching Pursuit

Publication Year: 2012, Page(s):1094 - 1121
Cited by:  Papers (616)  |  Patents (2)
| | PDF (6565 KB) | HTML

Finding the sparsest solution to underdetermined systems of linear equationsy= Φxis NP-hard in general. We show here that for systems with “typical”/“random” Φ, a good approximation to the sparsest solution is obtained by applying a fixed number of standard operations from linear algebra. Our proposal, Stagewise Orthogonal Matching Pursuit (StOMP), successively transforms the sig... View full abstract»

• ### Joint Spatial Division and Multiplexing—The Large-Scale Array Regime

Publication Year: 2013, Page(s):6441 - 6463
Cited by:  Papers (507)  |  Patents (4)
| | PDF (6169 KB) | HTML

We propose joint spatial division and multiplexing (JSDM), an approach to multiuser MIMO downlink that exploits the structure of the correlation of the channel vectors in order to allow for a large number of antennas at the base station while requiring reduced-dimensional channel state information at the transmitter (CSIT). JSDM achieves significant savings both in the downlink training and in the... View full abstract»

• ### The Optimal Hard Threshold for Singular Values is$4/\sqrt {3}$

Publication Year: 2014, Page(s):5040 - 5053
Cited by:  Papers (61)
| | PDF (2730 KB) | HTML

We consider recovery of low-rank matrices from noisy data by hard thresholding of singular values, in which empirical singular values below a threshold λ are set to 0. We study the asymptotic mean squared error (AMSE) in a framework, where the matrix size is large compared with the rank of the matrix to be recovered, and the signal-to-noise ratio of the low-rank piece stays constant. The AMSE-opti... View full abstract»

• ### Update or Wait: How to Keep Your Data Fresh

Publication Year: 2017, Page(s):7492 - 7508
Cited by:  Papers (29)
| | PDF (1178 KB) | HTML

In this paper, we study how to optimally manage the freshness of information updates sent from a source node to a destination via a channel. A proper metric for data freshness at the destination is the age-of-information, or simply age, which is defined as how old the freshest received update is, since the moment that this update was generated at the source node (e.g., a sensor). A reasonable upda... View full abstract»

• ### Signal Recovery From Random Measurements Via Orthogonal Matching Pursuit

Publication Year: 2007, Page(s):4655 - 4666
Cited by:  Papers (4226)  |  Patents (39)
| | PDF (936 KB) | HTML

This paper demonstrates theoretically and empirically that a greedy algorithm called orthogonal matching pursuit (OMP) can reliably recover a signal with m nonzero entries in dimension d given O(m ln d) random linear measurements of that signal. This is a massive improvement over previous results, which require <i>O</i>(m<sup>2</sup>) measurements. The new results for OMP a... View full abstract»

• ### A Fundamental Tradeoff Between Computation and Communication in Distributed Computing

Publication Year: 2018, Page(s):109 - 128
Cited by:  Papers (23)
| | PDF (1566 KB) | HTML

How can we optimally trade extra computing power to reduce the communication load in distributed computing? We answer this question by characterizing a fundamental tradeoff between computation and communication in distributed computing, i.e., the two are inversely proportional to each other. More specifically, a general distributed computing framework, motivated by commonly used structures like Ma... View full abstract»

• ### Fundamental Limits of Caching

Publication Year: 2014, Page(s):2856 - 2867
Cited by:  Papers (535)
| | PDF (783 KB) | HTML

Caching is a technique to reduce peak traffic rates by prefetching popular content into memories at the end users. Conventionally, these memories are used to deliver requested content in part from a locally cached copy rather than through the network. The gain offered by this approach, which we term local caching gain, depends on the local cache size (i.e., the memory available at each individual ... View full abstract»

• ### Theoretical Insights Into the Optimization Landscape of Over-Parameterized Shallow Neural Networks

Publication Year: 2019, Page(s):742 - 769
| | PDF (746 KB) | HTML

In this paper, we study the problem of learning a shallow artificial neural network that best fits a training data set. We study this problem in the over-parameterized regime where the numbers of observations are fewer than the number of parameters in the model. We show that with the quadratic activations, the optimization landscape of training, such shallow neural networks, has certain favorable ... View full abstract»

• ### Subspace Pursuit for Compressive Sensing Signal Reconstruction

Publication Year: 2009, Page(s):2230 - 2249
Cited by:  Papers (1076)  |  Patents (8)
| | PDF (976 KB) | HTML

We propose a new method for reconstruction of sparse signals with and without noisy perturbations, termed the subspace pursuit algorithm. The algorithm has two important characteristics: low computational complexity, comparable to that of orthogonal matching pursuit techniques when applied to very sparse signals, and reconstruction accuracy of the same order as that of linear programming (LP) opti... View full abstract»

• ### FemtoCaching: Wireless Content Delivery Through Distributed Caching Helpers

Publication Year: 2013, Page(s):8402 - 8413
Cited by:  Papers (404)
| | PDF (2158 KB) | HTML

Video on-demand streaming from Internet-based servers is becoming one of the most important services offered by wireless networks today. In order to improve the area spectral efficiency of video transmission in cellular systems, small cells heterogeneous architectures (e.g., femtocells, WiFi off-loading) are being proposed, such that video traffic to nomadic users can be handled by short-range lin... View full abstract»

• ### Design of capacity-approaching irregular low-density parity-check codes

Publication Year: 2001, Page(s):619 - 637
Cited by:  Papers (2026)  |  Patents (106)
| | PDF (510 KB) | HTML

We design low-density parity-check (LDPC) codes that perform at rates extremely close to the Shannon capacity. The codes are built from highly irregular bipartite graphs with carefully chosen degree patterns on both sides. Our theoretical analysis of the codes is based on the work of Richardson and Urbanke (see ibid., vol.47, no.2, p.599-618, 2000). Assuming that the underlying communication chann... View full abstract»

• ### The capacity of low-density parity-check codes under message-passing decoding

Publication Year: 2001, Page(s):599 - 618
Cited by:  Papers (1768)  |  Patents (154)
| | PDF (475 KB) | HTML

We present a general method for determining the capacity of low-density parity-check (LDPC) codes under message-passing decoding when used over any binary-input memoryless channel with discrete or continuous output alphabets. Transmitting at rates below this capacity, a randomly chosen element of the given ensemble will achieve an arbitrarily small target probability of error with a probability th... View full abstract»

• ### Massive MIMO Systems With Non-Ideal Hardware: Energy Efficiency, Estimation, and Capacity Limits

Publication Year: 2014, Page(s):7112 - 7139
Cited by:  Papers (302)
| | PDF (2212 KB) | HTML

The use of large-scale antenna arrays can bring substantial improvements in energy and/or spectral efficiency to wireless systems due to the greatly improved spatial resolution and array gain. Recent works in the field of massive multiple-input multiple-output (MIMO) show that the user channels decorrelate when the number of antennas at the base stations (BSs) increases, thus strong signal gains a... View full abstract»

• ### From Denoising to Compressed Sensing

Publication Year: 2016, Page(s):5117 - 5144
Cited by:  Papers (74)
| | PDF (2753 KB) | HTML

A denoising algorithm seeks to remove noise, errors, or perturbations from a signal. Extensive research has been devoted to this arena over the last several decades, and as a result, todays denoisers can effectively remove large amounts of additive white Gaussian noise. A compressed sensing (CS) reconstruction algorithm seeks to recover a structured signal acquired using a small number of randomiz... View full abstract»

## Aims & Scope

IEEE Transactions on Information Theory publishes papers concerned with the transmission, processing, and utilization of information.

Full Aims & Scope

## Meet Our Editors

Editor-in-Chief
Alexander Barg

Department of Electrical and Computer Engineering and the Institute for Systems Research, University of Maryland

email: abarg-ittrans@ece.umd.edu