IEEE Transactions on Information Theory

Filter Results

Displaying Results 1 - 25 of 61

Publication Year: 2012, Page(s):C1 - C4
| PDF (168 KB)
• IEEE Transactions on Information Theory publication information

Publication Year: 2012, Page(s): C2
| PDF (42 KB)
• Type Classes of Context Trees

Publication Year: 2012, Page(s):4077 - 4093
Cited by:  Papers (3)
| | PDF (4321 KB) | HTML

It is well known that a tree model does not always admit a finite-state machine (FSM) representation with the same (minimal) number of parameters. Therefore, known characterizations of type classes for FSMs do not apply, in general, to tree models. In this paper, the type class of a sequence with respect to a given context tree is studied. An exact formula is derived for the size of the class, ext... View full abstract»

• Minimax Pointwise Redundancy for Memoryless Models Over Large Alphabets

Publication Year: 2012, Page(s):4094 - 4104
Cited by:  Papers (8)
| | PDF (2910 KB) | HTML

We study the minimax pointwise redundancy of universal coding for memoryless models over large alphabets and present two main results. We first complete studies initiated in Orlitsky and Santhanam deriving precise asymptotics of the minimax pointwise redundancy for all ranges of the alphabet size relative to the sequence length. Second, we consider the minimax pointwise redundancy for a family of ... View full abstract»

• Entropy Density and Mismatch in High-Rate Scalar Quantization With Rényi Entropy Constraint

Publication Year: 2012, Page(s):4105 - 4116
Cited by:  Papers (1)
| | PDF (3783 KB) | HTML

Properties of scalar quantization with th power distortion and constrained Rényi entropy of order are investigated. For an asymptotically (high-rate) optimal sequence of quantizers, the contribution to the Rényi entropy due to source values in a fixed interval is identified in terms of the “entropy density” of the quantizer sequence. This extends results related to th... View full abstract»

• Information-Theoretic Limits of Selecting Binary Graphical Models in High Dimensions

Publication Year: 2012, Page(s):4117 - 4134
Cited by:  Papers (20)
| | PDF (5134 KB) | HTML

The problem of graphical model selection is to estimate the graph structure of a Markov random field given samples from it. We analyze the information-theoretic limitations of the problem of graph selection for binary Markov random fields under high-dimensional scaling, in which the graph size and the number of edges k, and/or the maximal node degree d, are allowed to increase to infinity as a fun... View full abstract»

• Estimation of Nonlinear Functionals of Densities With Confidence

Publication Year: 2012, Page(s):4135 - 4159
Cited by:  Papers (7)
| | PDF (10239 KB) | HTML

This paper introduces a class of k-nearest neighbor (k-NN) estimators called bipartite plug-in (BPI) estimators for estimating integrals of nonlinear functions of a probability density, such as Shannon entropy and Rényi entropy. The density is assumed to be smooth, have bounded support, and be uniformly bounded from below on this set. Unlike previous k-NN estimators of nonlinear density fu... View full abstract»

• On the Stability of Empirical Risk Minimization in the Presence of Multiple Risk Minimizers

Publication Year: 2012, Page(s):4160 - 4163
Cited by:  Papers (1)
| | PDF (1380 KB) | HTML

Recently, Kutin and Niyogi investigated several notions of algorithmic stability-a property of a learning map conceptually similar to continuity-showing that training stability is sufficient for consistency of empirical risk minimization (ERM) while distribution-free CV-stability is necessary and sufficient for having finite VC-dimension. This paper concerns a phase transition in the training stab... View full abstract»

• Interior-Point Methods for Full-Information and Bandit Online Learning

Publication Year: 2012, Page(s):4164 - 4175
Cited by:  Papers (2)
| | PDF (2325 KB) | HTML

We study the problem of predicting individual sequences with linear loss with full and partial (or bandit) feed- back. Our main contribution is the first efficient algorithm for the problem of online linear optimization in the bandit setting which achieves the optimal Õ(√(T)) regret. In addition, for the full-information setting, we give a novel regret minimization algorithm. These ... View full abstract»

• Discrete Sampling and Interpolation: Universal Sampling Sets for Discrete Bandlimited Spaces

Publication Year: 2012, Page(s):4176 - 4200
Cited by:  Papers (1)
| | PDF (6522 KB) | HTML

We study the problem of interpolating all values of a discrete signal f of length N when d <; N values are known, especially in the case when the Fourier transform of the signal is zero outside some prescribed index set J; these comprise the (generalized) bandlimited spaces BJ. The sampling pattern for f is specified by an index set I, and is said to be a universal sampling set if sa... View full abstract»

• Compressed Sensing With General Frames via Optimal-Dual-Based $ell _{1}$-Analysis

Publication Year: 2012, Page(s):4201 - 4214
Cited by:  Papers (33)
| | PDF (3315 KB) | HTML

Compressed sensing with sparse frame representations is seen to have much greater range of practical applications than that with orthonormal bases. In such settings, one approach to recover the signal is known as ℓ1-analysis. We expand in this paper the performance analysis of this approach by providing a weaker recovery condition than existing results in the literature. Our anal... View full abstract»

• Joint Detection and Estimation: Optimum Tests and Applications

Publication Year: 2012, Page(s):4215 - 4229
Cited by:  Papers (38)
| | PDF (3439 KB) | HTML

We consider a well-defined joint detection and parameter estimation problem. By combining the Bayesian formulation of the estimation subproblem with suitable constraints on the detection subproblem, we develop optimum one- and two-step test for the joint detection/estimation setup. The proposed combined strategies have the very desirable characteristic to allow for the trade-off between detection ... View full abstract»

• Estimating a Random Walk First-Passage Time From Noisy or Delayed Observations

Publication Year: 2012, Page(s):4230 - 4243
Cited by:  Papers (1)
| | PDF (3781 KB) | HTML

A Gaussian random walk (or a Wiener process), possibly with drift, is observed in a noisy or delayed fashion. The problem considered in this paper is to estimate the first time τ the random walk reaches a given level. Specifically, the average -moment (p ≥ 1 ) optimization problem infη E|η - τ|p is investigated where the infimum is taken ove... View full abstract»

• A Factor Graph Approach to Clock Offset Estimation in Wireless Sensor Networks

Publication Year: 2012, Page(s):4244 - 4260
Cited by:  Papers (17)
| | PDF (3283 KB) | HTML

The problem of clock offset estimation in a two-way timing message exchange regime is considered when the likelihood function of the observation time stamps is Gaussian, exponential, or log-normally distributed. A parameterized solution to the maximum likelihood (ML) estimation of clock offset is analytically obtained, which differs from the earlier approaches where the likelihood function is maxi... View full abstract»

• Counting Markov Types, Balanced Matrices, and Eulerian Graphs

Publication Year: 2012, Page(s):4261 - 4272
Cited by:  Papers (9)
| | PDF (3009 KB) | HTML

The method of types is one of the most popular techniques in information theory and combinatorics. Two sequences of equal length have the same type if they have identical empirical distributions. In this paper, we focus on Markov types, that is, sequences generated by a Markov source (of order one). We note that sequences having the same Markov type share the same so-called balanced frequency matr... View full abstract»

• Conditional Rényi Entropies

Publication Year: 2012, Page(s):4273 - 4277
Cited by:  Papers (19)
| | PDF (1360 KB) | HTML

There is no generally accepted definition of conditional Rényi entropy. The (unconditional) Rényi entropy depends on a parameter α, which for the case of min-entropy takes the value ∞. Even for this particular case, there are several proposals for the definition of conditional entropy. This paper describes three general definitions of conditional Rényi entropy... View full abstract»

• On the Equivalence Between Jensen–Shannon Divergence and Michelson Contrast

Publication Year: 2012, Page(s):4278 - 4288
Cited by:  Papers (10)
| | PDF (4810 KB) | HTML

This paper focuses on the link between a visible linear and local distortion in a pictorial scene and its cost in terms of information theory quantities. In particular, a formal relation between the Michelson visual contrast and the Jensen-Shannon divergence (JSD) will be provided. A universal just noticeable threshold is also derived by maximizing JSD constrained to the greatest Michelson contras... View full abstract»

• Interactive Source Coding for Function Computation in Collocated Networks

Publication Year: 2012, Page(s):4289 - 4305
Cited by:  Papers (28)
| | PDF (5921 KB) | HTML

A problem of interactive function computation in a collocated network is studied in a distributed block source coding framework. With the goal of computing samples of a desired function of sources at the sink, the source nodes exchange messages through a sequence of error-free broadcasts. For any function of independent sources, a computable characterization of the set of all feasible message codi... View full abstract»

• Bounds on the Capacity of Discrete Memoryless Channels Corrupted by Synchronization and Substitution Errors

Publication Year: 2012, Page(s):4306 - 4330
Cited by:  Papers (7)
| | PDF (4710 KB) | HTML

We study the capacity of discrete memoryless channels with synchronization errors and additive noise. We first show that with very large alphabets, their capacity can be achieved by independent and identically distributed input sources, and establish proven tight lower and upper capacity bounds. We also derive tight numerical capacity bounds for channels where the synchronization between the input... View full abstract»

• Opportunistic Capacity and Error Exponent Region for the Compound Channel With Feedback

Publication Year: 2012, Page(s):4331 - 4341
| | PDF (2905 KB) | HTML

Variable length communication over a compound channel with feedback is considered. Traditionally, capacity of a compound channel without feedback is defined as the maximum rate that is determined before the start of communication such that communication is reliable. This traditional definition is pessimistic. In the presence of feedback, an opportunistic definition is given. Capacity is defined as... View full abstract»

• Relaying in the Presence of Interference: Achievable Rates, Interference Forwarding, and Outer Bounds

Publication Year: 2012, Page(s):4342 - 4354
Cited by:  Papers (24)
| | PDF (3603 KB) | HTML

The smallest network model that captures relaying in the presence of multiple communicating pairs causing interference to each other is the interference channel with a relay. In this paper, an achievable rate region for the interference channel with a relay is derived. Special cases of strong interference under which this region is the capacity region are presented. The results obtained demonstrat... View full abstract»

• On the Structure of the Capacity Region of Asynchronous Memoryless Multiple-Access Channels

Publication Year: 2012, Page(s):4355 - 4366
| | PDF (3492 KB) | HTML

The asynchronous capacity region of memoryless multiple-access channels is the union of certain polytopes. It is well known that vertices of such polytopes may be approached via a technique called successive decoding. It is also known that an extension of successive decoding applies to the dominant face of such polytopes. The extension consists of forming groups of users in such a way that users w... View full abstract»

• Achievable Rates for $K$-User Gaussian Interference Channels

Publication Year: 2012, Page(s):4367 - 4380
Cited by:  Papers (13)
| | PDF (2766 KB) | HTML

The aim of this paper is to study the achievable rates for a K-user Gaussian interference channel (G-IFC) for any signal-to-noise ratio using a combination of lattice and algebraic codes. Lattice codes are first used to transform the G-IFC into a discrete input--output noiseless channel, and subsequently algebraic codes are developed to achieve good rates over this new alphabet. In this context, a... View full abstract»

• Aligned Interference Neutralization and the Degrees of Freedom of the 2$,times ,$2$,times ,$2 Interference Channel

Publication Year: 2012, Page(s):4381 - 4395
Cited by:  Papers (105)
| | PDF (2936 KB) | HTML

We show that the 2 × 2 × 2 interference network, i.e., the multihop interference network formed by concatenation of two two-user interference channels achieves the min-cut outer bound value of 2 DoF, for almost all values of channel coefficients, for both time-varying or fixed-channel coefficients. The key to this result is a new idea, called aligned interference neutralization, that... View full abstract»

• The Degrees of Freedom Region and Interference Alignment for the MIMO Interference Channel With Delayed CSIT

Publication Year: 2012, Page(s):4396 - 4417
Cited by:  Papers (81)
| | PDF (6183 KB) | HTML

The degrees of freedom (DoF) region of the two-user multiple-input multiple-output (MIMO) interference channel (IC) is studied under the assumptions of fast fading and delayed channel state information (CSI) at the transmitters (CSIT). Under our fast fading assumption, the channel matrices vary independently across time, so that delayed CSIT is equivalent to outdated CSIT. In particular, the DoF r... View full abstract»

Aims & Scope

IEEE Transactions on Information Theory publishes papers concerned with the transmission, processing, and utilization of information.

Full Aims & Scope

Meet Our Editors

Editor-in-Chief
Prakash Narayan

Department of Electrical and Computer Engineering