By Topic

# Information Theory, IEEE Transactions on

## Filter Results

Displaying Results 1 - 25 of 41

Page(s): C1 - C4
| | PDF (47 KB)
• ### IEEE Transactions on Information Theory publication information

Page(s): C2
| | PDF (38 KB)
• ### Classification With Finite Memory Revisited

Page(s): 4413 - 4421
| | | PDF (271 KB) |  | HTML

We consider the class of strong-mixing probability laws with positive transitions that are defined on doubly infinite sequences in a finite alphabet A. A device called the classifier (or discriminator) observes a training sequence whose probability law Q is unknown. The classifier's task is to consider a second probability law P and decide whether P = Q, or P and Q are sufficiently different according to some appropriate criterion Delta(Q,P) > Delta. If the classifier has available an infinite amount of training data, this is a simple matter. However, here we study the case where the amount of training data is limited to N letters. We define a function NDelta(Q|P), which quantifies the minimum length sequence needed to distinguish Q and P and the class M(NDelta) of all probability laws pairs (Q,P) that satisfy NDelta(Q|P) les NDelta for some given positive number NDelta. It is shown that every pair Q,P of probability laws that are sufficiently different according to the Delta criterion is contained in M(NDelta). We demonstrate that for any universal classifier there exists some Q for which the classification probability lambda(Q) = 1 for some N-sequence emerging from Q, for some P : (Q,P) epsi M circ(NDelta).Delta(Q,P) > Delta, if N < NDelta. Conversely, we introduce a classification algorithm that is essentially optimal in the sense that for every (Q,P) epsi M(NDelta), the probability of classification error lambda(Q) is uniformly vanishing with N for every P : (Q,P) epsi M circ(NDelta) if N ges NDelta 1+O(log log N Delta /log N Delta ). The proposed algorithm finds the largest empirical conditional divergence for a set of contexts which appear in the tested N-sequence. The computational complexity of the classification algorithm is O(N2(log N)3). Also, we introduce a second simplified context classification algorithm with a computational complexity of only O(N(log N)4) that is efficient in the sense that for every pair (Q,P) epsi M(NDelta), the pairwise probability of classification error lambda(Q,P) for the pair Q,P vanishes with N if N ges NDelta 1+O(log log N Delta /log N Delta ). Conversely, lambda(Q,P) = 1 at least for some (Q,P) epsi M(NDelta), if N < NDelta. View full abstract»

• ### Sufficient Conditions for Convergence of the Sum–Product Algorithm

Page(s): 4422 - 4437
| | | PDF (517 KB) |  | HTML

Novel conditions are derived that guarantee convergence of the sum-product algorithm (also known as loopy belief propagation or simply belief propagation (BP)) to a unique fixed point, irrespective of the initial messages, for parallel (synchronous) updates. The computational complexity of the conditions is polynomial in the number of variables. In contrast with previously existing conditions, our results are directly applicable to arbitrary factor graphs (with discrete variables) and are shown to be valid also in the case of factors containing zeros, under some additional conditions. The conditions are compared with existing ones, numerically and, if possible, analytically. For binary variables with pairwise interactions, sufficient conditions are derived that take into account local evidence (i.e., single-variable factors) and the type of pair interactions (attractive or repulsive). It is shown empirically that this bound outperforms existing bounds. View full abstract»

• ### Information conversion, effective samples, and parameter size

Page(s): 4438 - 4456
| | | PDF (747 KB) |  | HTML

Consider the relative entropy between a posterior density for a parameter given a sample and a second posterior density for the same parameter, based on a different model and a different data set. Then the relative entropy can be minimized over the second sample to get a virtual sample that would make the second posterior as close as possible to the first in an informational sense. If the first posterior is based on a dependent dataset and the second posterior uses an independence model, the effective inferential power of the dependent sample is transferred into the independent sample by the optimization. Examples of this optimization are presented for models with nuisance parameters, finite mixture models, and models for correlated data. Our approach is also used to choose the effective parameter size in a Bayesian hierarchical model. View full abstract»

• ### Joint Source–Channel Coding Error Exponent for Discrete Communication Systems With Markovian Memory

Page(s): 4457 - 4472
| | | PDF (681 KB) |  | HTML

We study the error exponent, EJ, for reliably transmitting a discrete stationary ergodic Markov (SEM) source Q over a discrete channel W with additive SEM noise via a joint source-channel (JSC) code. We first establish an upper bound for EJ in terms of the Renyi entropy rates of the source and noise processes. We next investigate the analytical computation of EJ by comparing our bound with Gallager's lower bound (1968) when the latter one is specialized to the SEM source-channel system. We also note that both bounds can be represented in Csiszar's form (1980), as the minimum of the sum of the source and channel error exponents. Our results provide us with the tools to systematically compare EJ with the tandem (separate) coding exponent EJ. We show that as in the case of memoryless source-channel pairs EJ les 2Er and we provide explicit conditions for which EJ > ET. Numerical results indicate that EJ ap 2ET for many SEM source-channel pairs, hence illustrating a substantial advantage of JSC coding over tandem coding for systems with Markovian memory. View full abstract»

• ### Sum Rate Characterization of Joint Multiple Cell-Site Processing

Page(s): 4473 - 4497
| | | PDF (612 KB) |  | HTML

• ### Capacity Results for Block-Stationary Gaussian Fading Channels With a Peak Power Constraint

Page(s): 4498 - 4520
| | | PDF (520 KB) |  | HTML

A peak-power-limited single-antenna block-stationary Gaussian fading channel is studied, where neither the transmitter nor the receiver knows the channel state information, but both know the channel statistics. This model subsumes most previously studied Gaussian fading models. The asymptotic channel capacity in the high signal-to-noise ratio (SNR) regime is first computed, and it is shown that the behavior of the channel capacity depends critically on the channel model. For the special case where the fading process is symbol-by-symbol stationary, it is shown that the codeword length must scale at least logarithmically with SNR in order to guarantee that the communication rate can grow logarithmically with SNR with decoding error probability bounded away from one. An expression for the capacity per unit energy is also derived. Furthermore, it is shown that the capacity per unit energy is achievable using temporal ON-OFF signaling with optimally allocated ON symbols, where the optimal ON-symbol allocation scheme may depend on the peak power constraint. View full abstract»

• ### Coding Problems for Channels With Partial State Information at the Transmitter

Page(s): 4521 - 4536
| | | PDF (346 KB) |  | HTML

Channel coding for single-user channels with rate- limited, coded, partial channel state information at the transmitter and full side information (SI) at the receiver is studied. In the first part of the current work, we consider joint source-channel state channel coding. In particular, we deal with lossy transmission of a source, over a cost-constrained state controlled channel where the receiver gets full SI and the transmitter receive coded partial SI. We derive a single-letter characterization of the achievable distortion-cost triples. From this characterization, a separation principle follows for both, coding of the main source and the transmitter SI. In the second part, we consider channel coding when the transmitter receives multiple partial, rate-limited descriptions of the state, and the receiver gets full SI. Two rate-limited descriptions of the state sequence are generated and conveyed to the transmitter, where each description can be lost independently during this transmission. For three different possible partial SI at the transmitter, we explore a channel coding strategy with three different forward channel rates. Inner and outer bounds are derived on the set of achievable partial description and forward channel rates. Furthermore, special cases where the inner bound is tight are studied. Similarities between coding of SI as multiple partial descriptions and the multiple description problem of source coding theory are pointed out. View full abstract»

• ### Distance Bounds for an Ensemble of LDPC Convolutional Codes

Page(s): 4537 - 4555
| | | PDF (611 KB) |  | HTML

An ensemble of (J, K) -regular low-density parity-check (LDPC) convolutional codes is introduced and existence-type lower bounds on the minimum distance dL, of code segments of finite length L and on the free distance dfree are derived. For sufficiently large constraint lengths v, the distances are shown to grow linearly with v and the ratio dL/v approaches the ratio dfee/v for large L. Moreover, the ratio of free distance to constraint length is several times larger than the ratio of minimum distance to block length for Gallager's ensemble of (J, K) -regular LDPC block codes. View full abstract»

• ### A Computationally Efficient Multilevel Coding Scheme for ISI Channels

Page(s): 4556 - 4566
| | | PDF (339 KB) |  | HTML

This paper proposes a multilevel coding scheme with linear mapping for intersymbol interference (ISI) channels and derives a low-complexity receiver structure that can achieve the ISI channel capacity. The transmitter superimposes many layers of independent binary antipodal streams to generate a quadrature amplitude modulation (QAM) or Gaussian-like channel input. The receiver performs multistage decoding with decision feedback and interference cancellation. Within each stage is a linear minimum mean-square-error (MMSE) equalizer followed by an error-correcting decoder. The complexity scales linearly with the channel length and the number of layers, and the process is shown to be asymptotically information lossless if a fixed input power is properly distributed over a sufficiently large number of layers. This framework is then extended to achieve the capacity of the ISI channel using a transmitter-side spectral shaping filter that converts a Gaussian input sequence with a white spectrum to one with a water-filling spectrum. View full abstract»

• ### Communication Over MIMO Broadcast Channels Using Lattice-Basis Reduction

Page(s): 4567 - 4582
| | | PDF (853 KB) |  | HTML

A new viewpoint for adopting the lattice reduction in communication over multiple-input multiple-output (MIMO) broadcast channels is introduced. Lattice basis reduction helps us to reduce the average transmitted energy by modifying the region which includes the constellation points. The new viewpoint helps us to generalize the idea of lattice-reduction-aided (LRA) preceding for the case of unequal-rate transmission, and obtain analytic results for the asymptotic behavior (signal-to-noise ratio (SNR) rarr infin) of the symbol error rate for the LRA precoding and the perturbation technique. Also, the outage probability for both cases of fixed-rate users and fixed sum rate is analyzed. It is shown that the LRA method, using the Lenstra-Lenstra-Lovasz (LLL) algorithm, achieves the optimum asymptotic slope of symbol error rate (called the precoding diversity). View full abstract»

• ### A Graph-Based Framework for Transmission of Correlated Sources Over Multiple-Access Channels

Page(s): 4583 - 4604
| | | PDF (808 KB) |  | HTML

In this paper, we consider a graph-based framework for transmission of correlated sources over multiple-access channels. It is well known that the separation approach is not optimal for this multiuser communication. Our objective in this work is to reintroduce modularity in this problem using a graph-based discrete interface and to minimize the performance loss as compared to the optimal joint source-channel coding scheme. The proposed framework envisages a transmission systems with two modules: a source-coding module and a channel-coding module. In the former module, the correlated sources are encoded distributively into correlated messages whose correlation structure can be associated with a bipartite graph. These correlated messages are then encoded by using correlated codewords and are reliably transmitted over the multiple-access channel in the latter module. This leads to performance gains in terms of enlarging the class of correlated sources that can be reliably transmitted over a multiple-access channel as compared to the conventional separation approach. We provide an information-theoretic characterization of 1) the rate of exponential growth (as a function of the number of channel uses) of the size of the bipartite graphs whose edges can be reliably transmitted over a multiuser channel and 2) the rate of exponential growth (as a function of the number of source samples) of the size of the bipartite graphs which can reliably represent a pair of correlated sources to be transmitted over a multiuser channel. View full abstract»

• ### Gallager Bounds for Noncoherent Decoders in Fading Channels

Page(s): 4605 - 4614
| | | PDF (318 KB) |  | HTML

Recently, Gallager's bounding techniques have been used to derive tight performance bounds for coded systems in fading channels. Most works in this field have thus far dealt with coherent decoding. This paper develops Gallager bounds for noncoherent systems in fading channels. Unlike coherent decoding, the exact error probability of a noncoherent decoder/detector conditioned on the fading coefficients does not admit a closed-form expression. This difficulty is overcome in this paper by employing the Chernoff technique. Although it weakens the bounds to some extent, the Chernoff technique enables the derivations of the limit-before-average (LBA) bound and Gallager bounds in closed form for noncoherent fading channels. Numerical examples show that the proposed bounds are convergent and are tighter than the conventional union bound. View full abstract»

• ### Performance Analysis of High Data Rate MIMO Systems in Frequency-Selective Fading Channels

Page(s): 4615 - 4627
| | | PDF (343 KB) |  | HTML

The performance of multicode direct-sequence spread-spectrum multiple-input multiple-output (MIMO) systems in the presence of frequency-selective fading is evaluated. We derive the asymptotic distribution of the multiple-antenna interference when the processing gain is sufficiently large. The probability of error is derived for the conventional RAKE receiver, and its performance is compared for various system configurations. We consider system tradeoffs for both fixed rate and fixed diversity. For a fixed total data rate, we demonstrate the advantage of decreasing the number of transmit antennas while increasing the number of codes, and for a fixed total diversity order, we demonstrate the advantage of decreasing the number of RAKE taps while increasing the number of receive antennas. View full abstract»

• ### High-Throughput Random Access Using Successive Interference Cancellation in a Tree Algorithm

Page(s): 4628 - 4639
| | | PDF (498 KB) |  | HTML

Random access is well motivated and has been widely applied when the network traffic is bursty and the expected throughput is not high. The main reason behind relatively low-throughput expectations is that collided packets are typically discarded. In this paper, we develop a novel protocol exploiting successive interference cancellation (SIC) in a tree algorithm (TA), where collided packets are reserved for reuse. Our SICTA protocol can achieve markedly higher maximum stable throughput relative to existing alternatives. Throughput performance is analyzed for general d-ary SICTA with both gated and window access. It is shown that the throughput for d-ary SICTA with gated access is about (ln d)/(d - 1), and can reach 0.693 for d = 2. This represents a 40% increase over the renowned first-come-first-serve (FCFS) 0.487 tree algorithm. Delay performance is also analyzed for SICTA with gated access, and numerical results are provided. View full abstract»

• ### Gossiping With Multiple Messages

Page(s): 4640 - 4654
| | | PDF (353 KB) |  | HTML

This paper investigates the dissemination of multiple pieces of information in large networks where users contact each other in a random uncoordinated manner, and users upload one piece per unit time. The underlying motivation is the design and analysis of piece selection protocols for peer-to-peer networks which disseminate files by dividing them into pieces. We first investigate one-sided protocols, where piece selection is based on the states of either the transmitter or the receiver. We show that any such protocol relying only on pushes, or alternatively only on pulls, is inefficient in disseminating all pieces to all users. We propose a hybrid one-sided piece selection protocol-INTERLEAVE-and show that by using both pushes and pulls it disseminates k pieces from a single source to n users in 9(k + log n) time, while obeying the constraint that each user can upload at most one piece in one unit of time, with high probability for large n. An optimal, unrealistic, centralized protocol would take k + log2 n time in this setting. For a soft upload constraint, the finishing time of INTERLEAVE is, with high probability, at most 3.2(k + log n). Moreover, efficient dissemination is also possible if the source implements forward erasure coding, and users push the latest released coded pieces (but do not pull). We also investigate two-sided protocols where piece selection is based on the states of both the transmitter and the receiver. We show that it is possible to disseminate n pieces to n users in n + O(log n) time, starting from an initial state where each user has a unique piece. View full abstract»

• ### Signal Recovery From Random Measurements Via Orthogonal Matching Pursuit

Page(s): 4655 - 4666
| | | PDF (936 KB) |  | HTML

This paper demonstrates theoretically and empirically that a greedy algorithm called orthogonal matching pursuit (OMP) can reliably recover a signal with m nonzero entries in dimension d given O(m ln d) random linear measurements of that signal. This is a massive improvement over previous results, which require O(m2) measurements. The new results for OMP are comparable with recent results for another approach called basis pursuit (BP). In some settings, the OMP algorithm is faster and easier to implement, so it is an attractive alternative to BP for signal recovery problems. View full abstract»

• ### Analytic Properties and Covariance Functions for a New Class of Generalized Gibbs Random Fields

Page(s): 4667 - 4679
| | | PDF (381 KB) |  | HTML

Spartan spatial random fields (SSRFs) are generalized Gibbs random fields, equipped with a coarse-graining kernel that acts as a low-pass filter for the fluctuations. SSRFs are defined by means of physically motivated spatial interactions and a small set of free parameters (interaction couplings). This paper focuses on the fluctuation-gradient-curvature (FGC) SSRF model, henceforth referred to as FGC-SSRF. This model is defined on the Euclidean space R by means of interactions proportional to the squares of the field realizations, as well as their gradient and curvature. The permissibility criteria of FGC-SSRFs are extended by considering the impact of a finite-bandwidth kernel. It is proved that the FGC-SSRFs are almost surely differentiable in the case of finite bandwidth. Asymptotic explicit expressions for the Spartan covariance function are derived for d = 1 and d= 3; both known and new covariance functions are obtained depending on the value of the FGC-SSRF shape parameter. Nonlinear dependence of the covariance integral scale on the FGC-SSRF characteristic length is established, and it is shown that the relation becomes linear asymptotically. The results presented in this paper are useful in random field parameter inference, and in spatial interpolation of irregularly spaced samples. View full abstract»

• ### Noise Covariance Properties in Dual-Tree Wavelet Decompositions

Page(s): 4680 - 4700
| | | PDF (882 KB) |  | HTML

Dual-tree wavelet decompositions have recently gained much popularity, mainly due to their ability to provide an accurate directional analysis of images combined with a reduced redundancy. When the decomposition of a random process is performed-which occurs in particular when an additive noise is corrupting the signal to be analyzed-it is useful to characterize the statistical properties of the dual-tree wavelet coefficients of this process. As dual-tree decompositions constitute over-complete frame expansions, correlation structures are introduced among the coefficients, even when a white noise is analyzed. In this paper, we show that it is possible to provide an accurate description of the covariance properties of the dual-tree coefficients of a wide-sense-stationary process. The expressions of the (cross-) covariance sequences of the coefficients are derived in the one- and two-dimensional cases. Asymptotic results are also provided, allowing to predict the behavior of the second-order moments for large lag values or at coarse resolution. In addition, the cross-correlations between the primal and dual wavelets, which play a primary role in our theoretical analysis, are calculated for a number of classical wavelet families. Simulation results are finally provided to validate these results. View full abstract»

• ### Performance Analysis of Linear Modulation Schemes With Generalized Diversity Combining on Rayleigh Fading Channels With Noisy Channel Estimates

Page(s): 4701 - 4727
| | | PDF (1598 KB) |  | HTML

Generalized diversity combining (GDC), also known as hybrid selection/maximal ratio combining or generalized selection combining, is a low-complexity diversity combining technique by which a fixed subset of a large number of available diversity channels is chosen and then combined using the rules of maximal ratio combining. In this paper, we analyze the performance of GDC on time-correlated Rayleigh fading channels with noisy channel estimates. We derive expressions for the probability of error for various linear modulation schemes with coherent detection, and discuss the conditions under which the analysis can be extended to noncoherent and differentially coherent receiver structures. Throughout the paper, using a fundamental approach to obtain the decision statistic at the combiner output, a number of new expressions for the error probabilities are obtained in a rigorous way, along with a presentation of their performance with channel estimation errors. The final expressions have roughly the same complexity of evaluation as that for the channel with only additive Gaussian noise. Our results correct various inaccuracies in the literature, and show that coherent receivers based on imperfectly estimated channel knowledge incur a significant performance loss. View full abstract»

• ### Modeling Block Decoding Approaches for the Fast Correlation Attack

Page(s): 4728 - 4737
| | | PDF (247 KB) |  | HTML

In this paper, a general framework which enables to compare previously proposed block decoding approaches for the fast correlation attack is developed. All attacks are based on decoding using a set of parity check sums of an underlying linear code. The purpose of this paper is twofold: 1) to provide a simple close form estimate about the number of check sums of a particular structure necessary for the corresponding attack to succeed; 2) to illustrate how such estimates are useful in minimizing the computational complexity of each attack considered, and consequently, in establishing a unified framework for comparison. View full abstract»

• ### Decoherence-Insensitive Quantum Communication by Optimal -Encoding

Page(s): 4738 - 4749
| | | PDF (295 KB) |  | HTML

The central issue in this paper is to transmit a quantum state in such a way that after some decoherence occurs, most of the information can be restored by a suitable decoding operation. For this purpose, we incorporate redundancy by mapping a given initial quantum state to a messenger state on a larger dimensional Hilbert space via a C* -algebra embedding. Our noise model for the transmission is a phase damping channel which admits a noiseless subsystem or decoherence-free subspace. More precisely, the transmission channel is obtained from convex combinations of a set of lowest rank yes/no measurements that leave a component of the messenger state unchanged. The objective of our encoding is to distribute quantum information optimally across the noise-susceptible component of the transmission when the noiseless component is not large enough to contain all the quantum information to be transmitted. We derive simple geometric conditions for optimal encoding and construct examples of such encodings. View full abstract»

• ### Blocking Bounds for Random Channel Selection on Tree Topologies

Page(s): 4750 - 4755
| | | PDF (288 KB) |  | HTML

Nonloopy network topologies are considered under the limitation that a channel can be utilized only if it is idle at all neighboring sites. Random channel selection is studied under a standard circuit-switched traffic model. Upper and lower bounds for blocking probabilities are determined via an auxiliary network process whose equilibrium distribution admits a computationally convenient form. By way of another approximate characterization, it is argued that random channel selection incurs vanishing loss of optimality as the number of channels and the traffic load increase in proportion. View full abstract»

• ### On the Linear Complexity and -Error Linear Complexity Over of the -ary Sidel'nikov Sequence

Page(s): 4755 - 4761
| | | PDF (285 KB) |  | HTML

The d-ary Sidel'nikov sequence S = s0, s1... of period q-1 for a prime power q=pm is a frequently analyzed sequence in the literature. Recently, it turned out that the linear complexity over Fp of the d-ary Sidel'nikov sequence is considerably smaller than the period if the sequence element s(q-1)/2mod(q-1) is chosen adequately. In this paper this work is continued and tight lower bounds on the linear complexity over Fp of the d-ary Sidel'nikov sequence are given. For certain cases exact values are provided. Finally, results on the k-error linear complexity over Fp of the d-ary Sidel'nikov sequence are presented. View full abstract»

## Aims & Scope

IEEE Transactions on Information Theory publishes papers concerned with the transmission, processing, and utilization of information.

Full Aims & Scope

## Meet Our Editors

Editor-in-Chief
Frank R. Kschischang

Department of Electrical and Computer Engineering