By Topic

Information Theory, IEEE Transactions on

Issue 11 • Date Nov. 2011

Filter Results

Displaying Results 1 - 25 of 39
  • Table of contents

    Page(s): C1 - C4
    Save to Project icon | Request Permissions | PDF file iconPDF (161 KB)  
    Freely Available from IEEE
  • IEEE Transactions on Information Theory publication information

    Page(s): C2
    Save to Project icon | Request Permissions | PDF file iconPDF (140 KB)  
    Freely Available from IEEE
  • Robust Matrix Decomposition With Sparse Corruptions

    Page(s): 7221 - 7234
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (341 KB) |  | HTML iconHTML  

    Suppose a given observation matrix can be decomposed as the sum of a low-rank matrix and a sparse matrix, and the goal is to recover these individual components from the observed sum. Such additive decompositions have applications in a variety of numerical problems including system identification, latent variable graphical modeling, and principal components analysis. We study conditions under which recovering such a decomposition is possible via a combination of 1 norm and trace norm minimization. We are specifically interested in the question of how many sparse corruptions are allowed so that convex programming can still achieve accurate recovery, and we obtain stronger recovery guarantees than previous studies. Moreover, we do not assume that the spatial pattern of corruptions is random, which stands in contrast to related analyses under such assumptions via matrix completion. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A Probabilistic and RIPless Theory of Compressed Sensing

    Page(s): 7235 - 7254
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (412 KB) |  | HTML iconHTML  

    This paper introduces a simple and very general theory of compressive sensing. In this theory, the sensing mechanism simply selects sensing vectors independently at random from a probability distribution F; it includes all standard models-e.g., Gaussian, frequency measurements-discussed in the literature, but also provides a framework for new measurement strategies as well. We prove that if the probability distribution F obeys a simple incoherence property and an isotropy property, one can faithfully recover approximately sparse signals from a minimal number of noisy measurements. The novelty is that our recovery results do not require the restricted isometry property (RIP) to hold near the sparsity level in question, nor a random model for the signal. As an example, the paper shows that a signal with s nonzero entries can be faithfully recovered from about s logn Fourier coefficients that are contaminated with noise. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • On the Performance of Sparse Recovery Via \ell _p -Minimization (0 \leq p \leq 1)

    Page(s): 7255 - 7278
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1199 KB) |  | HTML iconHTML  

    It is known that a high-dimensional sparse vector x* in TV can be recovered from low-dimensional measurements y = Ax* where Am×n(m <; n) is the measurement matrix. In this paper, with A being a random Gaussian matrix, we investigate the recovering ability of ℓp-minimization (0 ≤ p ≤ 1) as p varies, where ℓp-minimization returns a vector with the least ℓp quasi norm among all the vectors x satisfying Ax = y. Besides analyzing the performance of strong recovery where ℓp-minimization is re quired to recover all the sparse vectors up to certain sparsity, we also for the first time analyze the performance of "weak" recovery of ℓp -minimization (0 ≤ p <; 1) where the aim is to recover all the sparse vectors on one support with a fixed sign pattern. When α(:= m/n) → 1, we provide sharp thresholds of the sparsity ratio (i.e., percentage of nonzero entries of a vector) that differentiates the success and failure via ℓp -minimization. For strong recovery, the threshold strictly decreases from 0.5 to 0.239 as p increases from 0 to 1. Surprisingly, for weak recovery, the threshold is 2/3 for all p in [0, 1), while the threshold is 1 for ℓ1-minimization. We also explicitly demonstrate that ℓp-minimization (p <; 1) can re turn a denser solution than ℓp-minimization. For any a G (0.1), we provide bounds of the sparsity ratio for strong recovery and weak recovery, respectively, below which ℓp-minimization succeeds. Our bound of strong recovery improves on the existing bounds when a is large. In particular, regarding the recovery threshold, this paper argues that ℓp-minimization has a higher threshold with smaller p for strong recovery; the threshold is the same for all p for sectional - - recovery; and ℓp -minimization can outperform ℓp-minimization for weak recovery. These are in contrast to traditional wisdom that ℓp-minimization, though computationally more expensive, always has better sparse recovery ability than ℓ0 -minimization since it is closer to ℓ1-minimization. Finally, we provide an intuitive explanation to our findings. Numerical examples are also used to un ambiguously confirm and illustrate the theoretical predictions. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Optimally Sparse Frames

    Page(s): 7279 - 7287
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (192 KB) |  | HTML iconHTML  

    Frames have established themselves as a means to derive redundant, yet stable decompositions of a signal for analysis or transmission, while also promoting sparse expansions. However, when the signal dimension is large, the computation of the frame measurements of a signal typically requires a large number of additions and multiplications, and this makes a frame decomposition intractable in applications with limited computing budget. To address this problem, in this paper, we focus on frames in finite-dimensional Hilbert spaces and introduce sparsity for such frames as a new paradigm. In our terminology, a sparse frame is a frame whose elements have a sparse representation in an orthonormal basis, thereby enabling low-complexity frame decompositions. To introduce a precise meaning of optimality, we take the sum of the numbers of vectors needed from this orthonormal basis when expanding each frame vector as sparsity measure. We then analyze the recently introduced algorithm Spectral Tetris for construction of unit norm tight frames and prove that the tight frames generated by this algorithm are in fact optimally sparse with respect to the standard unit vector basis. Finally, we show that even the generalization of Spectral Tetris for the construction of unit norm frames associated with a given frame operator produces optimally sparse frames. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Inferring Rankings Using Constrained Sensing

    Page(s): 7288 - 7306
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (390 KB) |  | HTML iconHTML  

    We consider the problem of recovering a function over the space of permutations (or, the symmetric group) over n elements from given partial information; the partial information we consider is related to the group theoretic Fourier Transform of the function. This problem naturally arises in several settings such as ranked elections, multi-object tracking, ranking systems, and recommendation systems. Inspired by the work of Donoho and Stark in the context of discrete-time functions, we focus on non-negative functions with a sparse support (support size <;<; domain size). Our recovery method is based on finding the sparsest solution (through l0 optimization) that is consistent with the available information. As the main result, we derive sufficient conditions for functions that can be recovered exactly from partial information through l0 optimization. Under a natural random model for the generation of functions, we quantify the recoverability conditions by deriving bounds on the sparsity (support size) for which the function satisfies the sufficient conditions with a high probability as n → ∞. ℓ0 optimization is computationally hard. Therefore, the popular compressive sensing literature considers solving the convex relaxation, ℓ1 optimization, to find the sparsest solution. However, we show that ℓ1 optimization fails to recover a function (even with constant sparsity) generated using the random model with a high probability as n → ∞. In order to overcome this problem, we propose a novel iterative algorithm for the recovery of functions that satisfy the sufficient conditions. Finally, using an Information Theoretic framework, we study necessary conditions for exact recovery to be possible. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Derivative of Mutual Information at Zero SNR: The Gaussian-Noise Case

    Page(s): 7307 - 7312
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (167 KB) |  | HTML iconHTML  

    Assuming additive Gaussian noise, a general sufficient condition on the input distribution is established to guarantee that the ratio of mutual information to signal-to-noise ratio (SNR) goes to one half nat as SNR vanishes. The result allows SNR-dependent input distribution and side information. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Skewincidence

    Page(s): 7313 - 7316
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (115 KB) |  | HTML iconHTML  

    We introduce a new class of problems lying halfway between questions about graph capacity and intersection. We say that two binary sequences x and y of the same length have a skewincidence if there is a coordinate i for which xi=yi+1=1 or vice versa. We give relatively close bounds on the maximum number of binary sequences of length n any pair of which has a skewincidence. A systematic study of these problems helps to understand the mathematical difficulties to solve zero-error problems in information theory. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Probing Capacity

    Page(s): 7317 - 7332
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (711 KB) |  | HTML iconHTML  

    We consider the problem of optimal probing of states of a channel by transmitter and receiver for maximizing rate of reliable communication. The channel is discrete memoryless (DMC) with i.i.d. states. The encoder takes probing actions dependent on the message. It then uses the state information obtained from probing causally or noncausally to generate channel input symbols. The decoder may also take channel probing actions as a function of the observed channel output and use the channel state information thus acquired, along with the channel output, to estimate the message. We refer to the maximum achievable rate for reliable communication for such systems as the “Probing Capacity”. We characterize this capacity when the encoder and decoder actions are cost constrained. To motivate the problem, we begin by characterizing the trade-off between the capacity and fraction of channel states the encoder is allowed to observe, while the decoder is aware of channel states. In this setting of `to observe or not to observe' state at the encoder, we compute certain numerical examples which exhibit a pleasing phenomenon, where encoder can observe a relatively small fraction of states and yet communicate at maximum rate, i.e., rate when observing states at encoder is not cost constrained. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Communication Over Individual Channels

    Page(s): 7333 - 7358
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (5534 KB) |  | HTML iconHTML  

    A communication problem in considered, where no mathematical model is specified for the channel. The achievable rates are determined as a function of the channel input and output sequences known a-posteriori, without assuming any a-priori relation between them. For discrete channels the empirical mutual information between the input and output sequences is shown to be achievable, while for continuous channels the achievable rate is based on the empirical correlation between the sequences. A rate-adaptive scheme employing feedback which achieves these rates asymptotically with a guaranteed reliability, without prior knowledge of the channel behavior, is presented. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • The Capacity Region of p -Transmitter/ q -Receiver Multiple-Access Channels With Common Information

    Page(s): 7359 - 7376
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (5711 KB) |  | HTML iconHTML  

    This paper investigates the capacity problem for some multiple-access scenarios with cooperative transmitters. First, a general Multiple-Access Channel (MAC) with common information, i.e., a scenario where p transmitters send private messages and also a common message to q receivers and each receiver decodes all of the messages, is considered. The capacity region of the discrete memoryless channel is characterized. Then, the general Gaussian fading MAC with common information wherein partial Channel State Information (CSI) is available at the transmitters (CSIT) and perfect CSI is available at the receivers (CSIR) is investigated. A coding theorem is proved for this model that yields an exact characterization of the throughput capacity region. Finally, a two-transmitter/one-receiver Gaussian fading MAC with conferencing encoders with partial CSIT and perfect CSIR is studied and its capacity region is determined. For the Gaussian fading models with CSIR only (transmitters have no access to CSIT), some numerical examples and simulation results are provided for Rayleigh fading. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Noisy Channel Coding via Privacy Amplification and Information Reconciliation

    Page(s): 7377 - 7385
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (254 KB) |  | HTML iconHTML  

    We show that optimal protocols for noisy channel coding of public or private information over either classical or quantum channels can be directly constructed from two more primitive information-theoretic protocols: privacy amplification and information reconciliation, also known as data compression with side information. We do this in the one-shot scenario of structureless resources, and formulate our results in terms of the smooth min- and max-entropy. In the context of classical information theory, this shows that essentially all two-terminal protocols can be reduced to these two primitives, which are in turn governed by the smooth min- and max-entropies, respectively. In the context of quantum information theory, the recently-established duality of these two protocols means essentially all two-terminal protocols can be constructed using just a single primitive. As an illustration, we show how optimal noisy channel coding protocols can be constructed solely from privacy amplification. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Improved Linear Programming Decoding of LDPC Codes and Bounds on the Minimum and Fractional Distance

    Page(s): 7386 - 7402
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (851 KB) |  | HTML iconHTML  

    We examine LDPC codes decoded using linear programming (LP). Four contributions to the LP framework are presented. First, a new method of tightening the LP relaxation, and thus improving the LP decoder, is proposed. Second, we present an algorithm which calculates a lower bound on the minimum distance of a specific code. This algorithm exhibits complexity which scales quadratically with the block length. Third, we propose a method to obtain a tight lower bound on the fractional distance, also with quadratic complexity, and thus less than previously-existing methods. Finally, we show how the fundamental LP polytope for generalized LDPC codes and nonbinary LDPC codes can be obtained. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Coding for High-Density Recording on a 1-D Granular Magnetic Medium

    Page(s): 7403 - 7417
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (454 KB) |  | HTML iconHTML  

    In terabit-density magnetic recording, several bits of data can be replaced by the values of their neighbors in the storage medium. As a result, errors in the medium are dependent on each other and also on the data written. We consider a simple 1-D combinatorial model of this medium. In our model, we assume a setting where binary data is sequentially written on the medium and a bit can erroneously change to the immediately preceding value. We derive several properties of codes that correct this type of errors, focusing on bounds on their cardinality. We also define a probabilistic finite-state channel model of the storage medium, and derive lower and upper estimates of its capacity. A lower bound is derived by evaluating the symmetric capacity of the channel, i.e., the maximum transmission rate under the assumption of the uniform input distribution of the channel. An upper bound is found by showing that the original channel is a stochastic degradation of another, related channel model whose capacity we can compute explicitly. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Characteristic Generators and Dualization for Tail-Biting Trellises

    Page(s): 7418 - 7430
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (3206 KB) |  | HTML iconHTML  

    This paper focuses on dualizing tail-biting trellises, particularly KV trellises. These trellises are based on characteristic generators, as introduced by Koetter-Vardy (2003), and may be regarded as a natural generalization of minimal conventional trellises, even though they are not necessarily minimal. Two dualization techniques will be investigated: the local dualization, introduced by Forney (2001) for general normal graphs, and a linear-algebra-based dualization tailored to the specific class of tail-biting Bahl-Cocke-Jelinek-Raviv (BCJR) trellises, introduced by Nori-Shankar (2006). It turns out that, in general, the BCJR dual is a subtrellis of the local dual, while for KV trellises these two coincide. Furthermore, making use of both the BCJR construction and the local dualization, it will be shown that for each complete set of characteristic generators of a code there exists a complete set of characteristic generators of the dual code such that their resulting KV trellises are dual to each other if paired suitably. This proves a stronger version of a conjecture formulated by Koetter-Vardy. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Constant-Weight Gray Codes for Local Rank Modulation

    Page(s): 7431 - 7442
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (514 KB) |  | HTML iconHTML  

    We consider the local rank-modulation (LRM) scheme in which a sliding window going over a sequence of real-valued variables induces a sequence of permutations. LRM is a generalization of the rank-modulation scheme, which has been recently suggested as a way of storing information in flash memory. We study constant-weight Gray codes for the LRM scheme in order to simulate conventional multilevel flash cells while retaining the benefits of rank modulation. We present a practical construction of codes with asymptotically-optimal rate and weight asymptotically half the length, thus having an asymptotically-optimal charge difference between adjacent cells. Next, we turn to examine the existence of optimal codes by specifically studying codes of weight 2 and 3. In the former case, we upper bound the code efficiency, proving that there are no such asymptotically-optimal cyclic codes. In contrast, for the latter case we construct codes which are asymptotically-optimal. We conclude by providing necessary conditions for the existence of cyclic and cyclic optimal Gray codes. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Semibent Functions From Dillon and Niho Exponents, Kloosterman Sums, and Dickson Polynomials

    Page(s): 7443 - 7458
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (6102 KB) |  | HTML iconHTML  

    Kloosterman sums have recently become the focus of much research, most notably due to their applications in cryptography and coding theory. In this paper, we extensively investigate the link between the semibentness property of functions in univariate forms obtained via Dillon and Niho functions and Kloosterman sums. In particular, we show that zeros and the value four of binary Kloosterman sums give rise to semibent functions in even dimension with maximum degree. Moreover, we study the semibentness property of functions in polynomial forms with multiple trace terms and exhibit criteria involving Dickson polynomials. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Some Codes Correcting Asymmetric Errors of Limited Magnitude

    Page(s): 7459 - 7472
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (366 KB) |  | HTML iconHTML  

    An error model with asymmetric errors of limited magnitude is a good model for some multilevel flash memories. This paper is about constructions of codes correcting such errors. The main results are about codes correcting a single such error and codes of length m correcting all errors in m-1 or less positions. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Product Constructions for Perfect Lee Codes

    Page(s): 7473 - 7481
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (2056 KB) |  | HTML iconHTML  

    A well-known conjecture of Golomb and Welch is that the only nontrivial perfect codes in the Lee and Manhattan metrics have length two or minimum distance three. This problem and related topics were subject for extensive research in the last 40 years. In this paper, two product constructions for perfect Lee codes and diameter perfect Lee codes are presented. These constructions yield a large number of nonlinear perfect codes and nonlinear diameter perfect codes in the Lee and Manhattan metrics. A short survey and other related problems on perfect codes in the Lee and Manhattan metrics are also discussed. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Constant Composition Codes as Subcodes of Cyclic Codes

    Page(s): 7482 - 7488
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (2327 KB) |  | HTML iconHTML  

    Constant composition codes are codes where the frequency distribution of the elements in a codeword is the same for all codewords. In this paper, three classes of constant composition codes are constructed. These codes are subcodes of cyclic codes which have few weights occurring among the codewords. The new codes are excellent asymptotically compared to the previously best known constant composition codes. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Relative Difference Families With Variable Block Sizes and Their Related OOCs

    Page(s): 7489 - 7497
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (261 KB) |  | HTML iconHTML  

    Seven infinite classes of relative difference families with variable block sizes are presented explicitly. In particular, a balanced (gv,g,K,1)-DF with gk∈K[(k2-k)/2] is explicitly given for: (i) K={3,4,5} and every v coprime to 6; (ii) K={3,4,6}, {3,5,6} or {3,4,5,6} and every v coprime to 30. As far as the authors are aware, these difference families can be viewed as the first explicit constructions of infinite classes of optimal variable-weight optical orthogonal codes with more than two weights. It is observed, however, that there are infinitely many values of v for which an optimal (v,W,1,Q) -OOC exists, whatever the set of weights W and the weight distribution sequence Q are. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Binary Self-Dual Codes of Lengths 52 to 60 With an Automorphism of Order 7 or 13

    Page(s): 7498 - 7506
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (502 KB) |  | HTML iconHTML  

    All binary [n,n/2] optimal self-dual codes for length 52 ≤ n ≤ 60 with an automorphism of order 7 or 13 are classified up to equivalence. Two of the constructed [54,27,10] codes have weight enumerators that were not previously known to exist. There are also some [58,29,10] codes with new values of the parameters in their weight enumerator. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Weight Distributions of Regular Low-Density Parity-Check Codes Over Finite Fields

    Page(s): 7507 - 7521
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (359 KB) |  | HTML iconHTML  

    The average weight distribution of a regular low-density parity-check (LDPC) code ensemble over a finite field is thoroughly analyzed. In particular, a precise asymptotic approximation of the average weight distribution is derived for the small-weight case, and a series of fundamental qualitative properties of the asymptotic growth rate of the average weight distribution are proved. Based on this analysis, a general result, including all previous results as special cases, is established for the minimum distance of individual codes in a regular LDPC code ensemble. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • On the Per-Sample Capacity of Nondispersive Optical Fibers

    Page(s): 7522 - 7541
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (4139 KB) |  | HTML iconHTML  

    The capacity of the channel defined by the stochastic nonlinear Schrödinger equation, which includes the effects of the Kerr nonlinearity and amplified spontaneous emission noise, is considered in the case of zero dispersion. In the absence of dispersion, this channel behaves as a collection of parallel per-sample channels. The conditional probability density function of the nonlinear per-sample channels is derived using both a sum-product and a Fokker-Planck differential equation approach. It is shown that, for a fixed noise power, the per-sample capacity grows unboundedly with input signal. The channel can be partitioned into amplitude and phase subchannels, and it is shown that the contribution to the total capacity of the phase channel declines for large input powers. It is found that a 2-D distribution with a half-Gaussian profile on the amplitude and uniform phase provides a lower bound for the zero-dispersion optical fiber channel, which is simple and asymptotically capacity-achieving at high signal-to-noise ratios (SNRs). A lower bound on the capacity is also derived in the medium-SNR region. The exact capacity subject to peak and average power constraints is numerically quantified using dense multiple ring modulation formats. The differential model underlying the zero-dispersion channel is reduced to an algebraic model, which is more tractable for digital communication studies, and, in particular, it provides a relation between the zero-dispersion optical channel and a 2 × 2 multiple-input multiple-output Rician fading channel. It appears that the structure of the capacity-achieving input distribution resembles that of the Rician fading channel, i.e., it is discrete in amplitude with a finite number of mass points, while continuous and uniform in phase. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.

Aims & Scope

IEEE Transactions on Information Theory publishes papers concerned with the transmission, processing, and utilization of information.

Full Aims & Scope

Meet Our Editors

Editor-in-Chief
Frank R. Kschischang

Department of Electrical and Computer Engineering