By Topic

Information Theory, IEEE Transactions on

Issue 9 • Date Sept. 2013

Filter Results

Displaying Results 1 - 25 of 69
  • Table of contents

    Publication Year: 2013 , Page(s): C1 - C4
    Save to Project icon | Request Permissions | PDF file iconPDF (179 KB)  
    Freely Available from IEEE
  • IEEE Transactions on Information Theory publication information

    Publication Year: 2013 , Page(s): C2
    Save to Project icon | Request Permissions | PDF file iconPDF (142 KB)  
    Freely Available from IEEE
  • Fundamental Limits of Cooperation

    Publication Year: 2013 , Page(s): 5213 - 5226
    Cited by:  Papers (39)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (2077 KB) |  | HTML iconHTML  

    Cooperation is viewed as a key ingredient for interference management in wireless networks. This paper shows that cooperation has fundamental limitations. First, it is established that in systems that rely on pilot-assisted channel estimation, the spectral efficiency is upper-bounded by a quantity that does not depend on the transmit powers; in this framework, cooperation is possible only within clusters of limited size, which are subject to out-of-cluster interference whose power scales with that of the in-cluster signals. Second, an upper bound is also shown to exist if the cooperation extends to an entire (large) system operating as a single cluster; here, pilot-assisted transmission is necessarily transcended. Altogether, it is concluded that cooperation cannot in general change an interference-limited network to a noise-limited one. Consequently, the existing literature that routinely assumes that the high-power spectral efficiency scales with the log-scale transmit power provides only a partial characterization. The complete characterization proposed in this paper subdivides the high-power regime into a degree-of-freedom regime, where the scaling with the log-scale transmit power holds approximately, and a saturation regime, where the spectral efficiency hits a ceiling that is independent of the power. Using a cellular system as an example, it is demonstrated that the spectral efficiency saturates at power levels of operational relevance. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Compute-and-Forward Strategies for Cooperative Distributed Antenna Systems

    Publication Year: 2013 , Page(s): 5227 - 5243
    Cited by:  Papers (16)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (3304 KB) |  | HTML iconHTML  

    We study a distributed antenna system where L antenna terminals (ATs) are connected to a central processor (CP) via digital error-free links of finite capacity R0, and serve K user terminals (UTs). This model has been widely investigated both for the uplink (UTs to CP) and for the downlink (CP to UTs), which are instances of the general multiple-access relay and broadcast relay networks. We contribute to the subject in the following ways: 1) For the uplink, we consider the recently proposed “compute and forward” (CoF) approach and examine the corresponding system optimization at finite SNR. 2) For the downlink, we propose a novel precoding scheme nicknamed “reverse compute and forward” (RCoF). 3) In both cases, we present low-complexity versions of CoF and RCoF based on standard scalar quantization at the receivers, that lead to discrete-input discrete-output symmetric memoryless channel models for which near-optimal performance can be achieved by standard single-user linear coding. 4) We provide extensive numerical results and finite SNR comparison with other “state of the art” information theoretic techniques, in scenarios including fading and shadowing. The proposed uplink and downlink system optimization focuses specifically on the ATs and UTs selection problem. In both cases, for a given set of transmitters, the goal consists of selecting a subset of the receivers such that the corresponding system matrix has full rank and the sum rate is maximized. We present low-complexity ATs and UTs selection schemes and demonstrate through Monte Carlo simulation that the proposed schemes essentially eliminate the problem of rank deficiency of the system matrix and greatly mitigate the noninteger penalty affecting CoF/RCoF at high SNR. Comparison with other state-of-the art information theoretic schemes, show competitive performance of the proposed approaches with significantly lower complexity. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Secrecy Degrees of Freedom of MIMO Broadcast Channels With Delayed CSIT

    Publication Year: 2013 , Page(s): 5244 - 5256
    Cited by:  Papers (8)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (4453 KB) |  | HTML iconHTML  

    The degrees of freedom (DoF) of the two-user Gaussian multiple-input and multiple-output (MIMO) broadcast channel with confidential messages is studied under the assumption that delayed channel state information (CSI) is available at the transmitter. We characterize the optimal secrecy DoF (SDoF) region and show that it can be achieved by a simple artificial noise alignment scheme. The proposed scheme sends the confidential messages superposed with the artificial noise over several time slots. Exploiting delayed CSI, the transmitter aligns the transmit signal in such a way that the useful message can be extracted at the intended receiver but is completely drowned by the artificial noise at the unintended receiver. The proposed scheme can be regarded as a nontrivial extension of Maddah-Ali Tse scheme and enables us to quantify the resource overhead, or equivalently the DoF loss, to be paid for the secure communications. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Ergodic Sum Capacity of Macrodiversity MIMO Systems in Flat Rayleigh Fading

    Publication Year: 2013 , Page(s): 5257 - 5270
    Cited by:  Papers (1)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (3918 KB) |  | HTML iconHTML  

    The prospect of base station cooperation leading to joint combining at widely separated antennas has led to increased interest in macrodiversity systems, where both sources and receive antennas are geographically distributed. In this scenario, analytical investigation of channel capacity is extremely challenging for finite-size systems since the channel matrices have a very general form where each path may have a different power. Hence, in this paper, we consider the ergodic sum capacity of a macrodiversity multiple-input multiple-output system with arbitrary numbers of sources and receive antennas operating over Rayleigh fading channels. For this system, we compute the exact ergodic capacity for a system with at most two transmit antennas and a compact approximation for the general system, which is shown to be very accurate over a wide range of cases. Finally, we compare our results with previous asymptotic results and bounds. Results are verified by Monte Carlo simulations and the impact on capacity of various channel power profiles is investigated. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • On the Distribution of MIMO Mutual Information: An In-Depth Painlevé-Based Characterization

    Publication Year: 2013 , Page(s): 5271 - 5296
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (5824 KB) |  | HTML iconHTML  

    This paper builds upon our recent work which computed the moment generating function of the multiple-input multiple-output mutual information exactly in terms of a Painlevé V differential equation. By exploiting this key analytical tool, we provide an in-depth characterization of the mutual information distribution for sufficiently large (but finite) antenna numbers. In particular, we derive systematic closed-form expansions for the high-order cumulants. These results yield considerable new insight, such as providing a technical explanation as to why the well-known Gaussian approximation is quite robust to large signal-to-noise ratio for the case of unequal antenna arrays, while it deviates strongly for equal antenna arrays. In addition, by drawing upon our high-order cumulant expansions, we employ the Edgeworth expansion technique to propose a refined Gaussian approximation which is shown to give a very accurate closed-form characterization of the mutual information distribution, both around the mean and for moderate deviations into the tails (where the Gaussian approximation fails remarkably). For stronger deviations where the Edgeworth expansion becomes unwieldy, we employ the saddle point method and asymptotic integration tools to establish new analytical characterizations which are shown to be very simple and accurate. Based on these results, we also recover key well-established properties of the tail distribution, including the diversity-multiplexing-tradeoff. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Wireless MIMO Switching: Weighted Sum Mean Square Error and Sum Rate Optimization

    Publication Year: 2013 , Page(s): 5297 - 5312
    Cited by:  Papers (9)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (4779 KB) |  | HTML iconHTML  

    This paper addresses joint transceiver and relay design for a wireless multiple-input multiple-output (MIMO) switching scheme that enables data exchange among multiple users. Here, a multiantenna relay linearly precodes the received (uplink) signals from multiple users and forwards the signal in the downlink, where the purpose of precoding is to let each user receive its desired signal with interference from other users suppressed. The problem of optimizing the precoder based on various design criteria is typically nonconvex and difficult to solve. The main contribution of this paper is a unified approach to solve the weighted sum mean square error (MSE) minimization and weighted sum rate maximization problems in MIMO switching. Specifically, an iterative algorithm is proposed for jointly optimizing the relay's precoder and the users' receive filters to minimize the weighted sum MSE. It is also shown that the weighted sum rate maximization problem can be reformulated as an iterated weighted sum MSE minimization problem and can, therefore, be solved similarly to the case of weighted sum MSE minimization. With properly chosen initial values, the proposed iterative algorithms are asymptotically optimal in both high- and low-signal-to-noise-ratio regimes for MIMO switching, either with or without self-interference cancellation (a.k.a., physical-layer network coding). Numerical results show that the optimized MIMO switching scheme based on the proposed algorithms significantly outperforms existing approaches in the literature. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Maximum-Rate Transmission With Improved Diversity Gain for Interference Networks

    Publication Year: 2013 , Page(s): 5313 - 5330
    Cited by:  Papers (2)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (3809 KB) |  | HTML iconHTML  

    Interference alignment (IA) was shown to be effective for interference management in improving transmission rate in terms of the degree of freedom (DoF) gain. On the other hand, orthogonal space-time block codes were widely used in point-to-point multiantenna channels to enhance transmission reliability in terms of the diversity gain. In this paper, we connect these two ideas, i.e., IA and space-time block coding, to improve the designs of alignment precoders for multiuser networks. Specifically, we consider the use of Alamouti codes for IA because of their rate-one transmission and achievability of full diversity in point-to-point systems. The Alamouti codes protect the desired link by introducing orthogonality between the two symbols in one Alamouti codeword, and create alignment at the interfering receiver. We show that the proposed alignment methods can maintain the maximum DoF gain and improve the ergodic mutual information in the long-term regime, while increasing the diversity gain to 2 in the short-term regime. The presented examples of interference networks have two antennas at each node and include the two-user X channel, the interfering multiaccess channel, and the interfering broadcast channel. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Joint Optimization of the Transmit Covariance and Relay Precoder in General Gaussian Amplify-and-Forward Relay Channels

    Publication Year: 2013 , Page(s): 5331 - 5351
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (5082 KB) |  | HTML iconHTML  

    The maximum data rate that can be achieved by the strictly causal full-duplex amplify-and-forward (AF) scheme in general Gaussian relay channels is achieved by Gaussian codebooks and can be cast as the solution of an optimization problem of the input transmit covariance and relay precoder. This problem possesses an intricate nonconvex structure and is hence difficult to solve. To circumvent this difficulty, the relay precoder is assumed to be given and then the Karush-Kuhn-Tucker conditions are used to obtain closed form expressions for the optimal input covariance corresponding to that precoder. These expressions are used to show that subdiagonal precoders suffice to attain the maximum achievable rate of the AF scheme at any source transmit power. In addition to significantly reducing the effort expended in searching for the optimal relay precoder, this observation enables us to find the optimal precoders at low and high source transmit powers. For asymptotically low transmit powers, the optimal relaying mechanism is shown to possess an interlacing structure, thereby resembling half-duplex operation. In contrast, for asymptotically high transmit powers, it is optimal for the relay to be silent. The asymptotic analysis enables us to develop an explicit formulation for a suboptimal precoder that, at intermediate source transmit powers, are shown numerically to outperform asymptotically optimal precoders. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Self-Synchronizing Pulse Position Modulation With Error Tolerance

    Publication Year: 2013 , Page(s): 5352 - 5362
    Cited by:  Papers (1)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1856 KB) |  | HTML iconHTML  

    Pulse position modulation (PPM) is a popular signal modulation technique which converts signals into M-ary data by means of the position of a pulse within a time interval. While PPM and its variations have great advantages in many contexts, this type of modulation is vulnerable to loss of synchronization, potentially causing a severe error floor or throughput penalty even when little or no noise is assumed. Another disadvantage is that this type of modulation typically offers no error correction mechanism on its own, making them sensitive to intersymbol interference and environmental noise. In this paper, we propose a coding theoretic variation of PPM that allows for significantly more efficient symbol and frame synchronization as well as strong error correction. The proposed scheme can be divided into a synchronization layer and a modulation layer. This makes our technique compatible with major existing techniques such as standard PPM, multipulse PPM, and expurgated PPM as well in that the scheme can be realized by adding a simple synchronization layer to one of these standard techniques. We also develop a generalization of expurgated PPM suited for the modulation layer of the proposed self-synchronizing modulation scheme. This generalized PPM can also be used as stand-alone error-correcting PPM with a larger number of available symbols. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • How Many Queries Will Resolve Common Randomness?

    Publication Year: 2013 , Page(s): 5363 - 5378
    Cited by:  Papers (3)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (4243 KB) |  | HTML iconHTML  

    A set of m terminals, observing correlated signals, communicate interactively to generate common randomness for a given subset of them. Knowing only the communication, how many direct queries of the value of the common randomness will resolve it? A general upper bound, valid for arbitrary signal alphabets, is developed for the number of such queries by using a query strategy that applies to all common randomness and associated communication. When the underlying signals are independent and identically distributed repetitions of m correlated random variables, the number of queries can be exponential in signal length. For this case, the mentioned upper bound is tight and leads to a single-letter formula for the largest query exponent, which coincides with the secret key capacity of a corresponding multiterminal source model. In fact, the upper bound constitutes a strong converse for the optimum query exponent, and implies also a new strong converse for secret key capacity. A key tool, estimating the size of a large probability set in terms of Rényi entropy, is interpreted separately, too, as a lossless block coding result for general sources. As a particularization, it yields the classic result for a discrete memoryless source. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Secrecy Outage Capacity of Fading Channels

    Publication Year: 2013 , Page(s): 5379 - 5397
    Cited by:  Papers (3)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (6100 KB) |  | HTML iconHTML  

    This paper considers point-to-point secure communication over flat fading channels under an outage constraint. More specifically, we extend the definition of outage capacity to account for the secrecy constraint and obtain sharp characterizations of the corresponding fundamental limits under two different assumptions on the transmitter channel state information (CSI). First, we find the outage secrecy capacity assuming that the transmitter has perfect knowledge of the legitimate and eavesdropper channel gains. In this scenario, the capacity achieving scheme relies on opportunistically exchanging private keys between the legitimate nodes. These keys are stored in a key buffer and later used to secure delay sensitive data using the Vernam's one time pad technique. We then extend our results to the more practical scenario where the transmitter is assumed to know only the legitimate channel gain. Here, our achievability arguments rely on privacy amplification techniques to generate secret key bits. In the two cases, we also characterize the optimal power control policies which, interestingly, turn out to be a judicious combination of channel inversion and the optimal ergodic strategy. Finally, we analyze the effect of key buffer overflow on the overall outage probability. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Capacity Region of the Asynchronous Gaussian Vector Multiple-Access Channel

    Publication Year: 2013 , Page(s): 5398 - 5420
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (7343 KB) |  | HTML iconHTML  

    In this paper, we derive explicit expressions for the capacity region of the two-user symbol-asynchronous Gaussian vector multiple-access channel. Verdú considered the case where each user linearly modulates a fixed waveform in each symbol period, where the symbol periods for the users are not perfectly aligned at the receiver. He derived explicit capacity region expressions for the case where the transmitters have knowledge of the mutual offset and also for the case where the transmitters have no knowledge of the mutual offset. In this paper, we extend Verdú's results to allow each user to linearly modulate a set of orthonormal waveforms, instead of a single waveform, in each symbol period and with no restrictions imposed on the waveforms. We consider group power constraints, which include individual sum power constraints, as orthonormal waveforms assigned to each user may come from different frequency bands with different power constraints. Similar to the case where each user is allowed to linearly modulate only a single waveform, our results hold regardless of whether or not the transmitters are frame synchronous. In addition, we present some results that are necessary to numerically compute the capacity region expressions with general purpose convex optimization algorithms. Next, we simplify the capacity region expression when there are only individual sum power constraints and when the transmitters know the mutual offset. We also prove a sufficient condition for a similar simplification to hold when the transmitters have no knowledge of the mutual offset. Finally, we consider a specialized algorithm to numerically compute the simplified capacity region expression when the transmitters know the mutual offset. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • At Low SNR, Asymmetric Quantizers are Better

    Publication Year: 2013 , Page(s): 5421 - 5445
    Cited by:  Papers (5)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (6545 KB) |  | HTML iconHTML  

    We study the capacity of the discrete-time Gaussian channel when its output is quantized with a 1-bit quantizer. We focus on the low signal-to-noise ratio (SNR) regime, where communication at very low spectral efficiencies takes place. In this regime, a symmetric threshold quantizer is known to reduce channel capacity by a factor of 2/π, i.e., to cause an asymptotic power loss of approximately 2 dB. Here, it is shown that this power loss can be avoided by using asymmetric threshold quantizers and asymmetric signaling constellations. To avoid this power loss, flash-signaling input distributions are essential. Consequently, 1-bit output quantization of the Gaussian channel reduces spectral efficiency. Threshold quantizers are not only asymptotically optimal: at every fixed SNR, a threshold quantizer maximizes capacity among all 1-bit output quantizers. The picture changes on the Rayleigh-fading channel. In the noncoherent case, a 1-bit output quantizer causes an unavoidable low-SNR asymptotic power loss. In the coherent case, however, this power loss is avoidable provided that we allow the quantizer to depend on the fading level. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Source-Channel Coding Theorems for the Multiple-Access Relay Channel

    Publication Year: 2013 , Page(s): 5446 - 5465
    Cited by:  Papers (4)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (6551 KB) |  | HTML iconHTML  

    We study reliable transmission of arbitrarily correlated sources over multiple-access relay channels (MARCs) and multiple-access broadcast relay channels (MABRCs). In MARCs only the destination is interested in reconstructing the sources, while in MABRCs, both the relay and the destination want to reconstruct them. In addition to arbitrary correlation among the source signals at the users, both the relay and the destination have side information correlated with the source signals. Our objective is to determine whether a given pair of sources can be losslessly transmitted to the destination for a given number of channel symbols per source sample, defined as the source-channel rate. Sufficient conditions for reliable communication based on operational separation, as well as necessary conditions on the achievable source-channel rates are characterized. Since operational separation is generally not optimal for MARCs and MABRCs, sufficient conditions for reliable communication using joint source-channel coding schemes based on a combination of the correlation preserving mapping technique with Slepian-Wolf source coding are also derived. For correlated sources transmitted over fading Gaussian MARCs and MABRCs, we present conditions under which separation (i.e., separate and stand-alone source and channel codes) is optimal. This is the first time optimality of separation is proved for MARCs and MABRCs. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Secure Lossy Transmission of Vector Gaussian Sources

    Publication Year: 2013 , Page(s): 5466 - 5487
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (6531 KB) |  | HTML iconHTML  

    We study the secure lossy transmission of a vector Gaussian source to a legitimate user in the presence of an eavesdropper, where both the legitimate user and the eavesdropper have vector Gaussian side information. The aim of the transmitter is to describe the source to the legitimate user in a way that the legitimate user can reconstruct the source within a certain distortion level while the eavesdropper is kept ignorant of the source as much as possible as measured by the equivocation. We obtain an outer bound for the rate, equivocation and distortion region of this secure lossy transmission problem. This outer bound is tight when the transmission rate constraint is removed. In other words, we obtain the maximum equivocation at the eavesdropper when the legitimate user needs to reconstruct the source within a fixed distortion level while there is no constraint on the transmission rate. This characterization of the maximum equivocation involves two auxiliary random variables. We show that a nontrivial selection for both random variables may be necessary in general. The necessity of two auxiliary random variables also implies that, in general, Wyner-Ziv coding is suboptimal in the presence of an eavesdropper. In addition, we show that, even when there is no rate constraint on the legitimate link, uncoded transmission (deterministic or stochastic) is suboptimal; the presence of an eavesdropper necessitates the use of a coded scheme to attain the maximum equivocation. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Universal Communication—Part I: Modulo Additive Channels

    Publication Year: 2013 , Page(s): 5488 - 5510
    Cited by:  Papers (2)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (4388 KB) |  | HTML iconHTML  

    Which communication rates can be attained over a channel whose output is an unknown (possibly stochastic) function of the input that may vary arbitrarily in time with no a priori model? Following the spirit of the finite-state compressibility of a sequence, defined by Lempel and Ziv, a “capacity” is defined for such a channel as the highest rate achievable by a designer knowing the particular relation that indeed exists between the input and output for all times, yet is constrained to use a fixed finite-length block communication scheme without feedback, i.e., use the same encoder and decoder over each block. In the case of the modulo additive channel, where the output sequence is obtained by modulo addition of an unknown individual sequence to the input sequence, this capacity is upper bounded by a function of the finite state compressibility of the noise sequence. A universal communication scheme with feedback that attains this capacity universally, without prior knowledge of the noise sequence, is presented. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Successive Refinement With Decoder Cooperation and Its Channel Coding Duals

    Publication Year: 2013 , Page(s): 5511 - 5533
    Cited by:  Papers (2)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (6678 KB) |  | HTML iconHTML  

    We study cooperation in multiterminal source coding models involving successive refinement. Specifically, we study the case of a single encoder and two decoders, where the encoder provides a common description to both the decoders and a private description to only one of the decoders. The decoders cooperate via cribbing, i.e., the decoder with access only to the common description is allowed to observe, in addition, a deterministic function of the reconstruction symbols produced by the other. We characterize the fundamental performance limits in the respective settings of noncausal, strictly causal, and causal cribbing. We use a coding scheme, referred to as Forward Encoding and Block Markov Decoding, which builds on one recently used by Cuff and Zhao for coordination via implicit communication. Finally, we use the insight gained to introduce and solve some dual-channel coding scenarios involving multiple-access channels with cribbing. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Bounds on the Capacity of Random Insertion and Deletion-Additive Noise Channels

    Publication Year: 2013 , Page(s): 5534 - 5546
    Cited by:  Papers (4)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (3986 KB) |  | HTML iconHTML  

    We develop several analytical lower bounds on the capacity of binary insertion and deletion channels by considering independent uniformly distributed (i.u.d.) inputs and computing lower bounds on the mutual information between the input and output sequences. For the deletion channel, we consider two different models: i.i.d. deletion-substitution channel and i.i.d. deletion channel with additive white Gaussian noise (AWGN). These two models are considered to incorporate effects of the channel noise along with the synchronization errors. For the insertion channel case, we consider Gallager's model in which the transmitted bits are replaced with two random bits and uniform over the four possibilities independently of any other insertion events. The general approach taken is similar in all cases, however the specific computations differ. Furthermore, the approach yields a useful lower bound on the capacity for a wide range of deletion probabilities of the deletion channels, while it provides a beneficial bound only for small insertion probabilities (less than 0.25) of the insertion model adopted. We emphasize the importance of these results by noting that: 1) our results are the first analytical bounds on the capacity of deletion-AWGN channels, 2) the results developed are the best available analytical lower bounds on the deletion-substitution case, 3) for the Gallager insertion channel model, the new lower bound improves the existing results for small insertion probabilities. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • First-Passage Time and Large-Deviation Analysis for Erasure Channels With Memory

    Publication Year: 2013 , Page(s): 5547 - 5565
    Cited by:  Papers (1)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (3415 KB) |  | HTML iconHTML  

    This paper considers the performance of digital communication systems transmitting messages over finite-state erasure channels with memory. Information bits are protected from channel erasures using error-correcting codes; successful receptions of codewords are acknowledged at the source through instantaneous feedback. The primary focus of this research is on delay-sensitive applications, codes with finite block lengths, and, necessarily, nonvanishing probabilities of decoding failure. The contribution of this paper is twofold. A methodology to compute the distribution of the time required to empty a buffer is introduced. Based on this distribution, the mean hitting time to an empty queue and delay-violation probabilities for specific thresholds can be computed explicitly. The proposed techniques apply to situations where the transmit buffer contains a predetermined number of information bits at the onset of the data transfer. Furthermore, as additional performance criteria, large deviation principles are obtained for the empirical mean service time and the average packet-transmission time associated with the communication process. This rigorous framework yields a pragmatic methodology to select code rate and block length for the communication unit as functions of the service requirements. Examples motivated by practical systems are provided to further illustrate the applicability of these techniques. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Universal Decoding for Arbitrary Channels Relative to a Given Class of Decoding Metrics

    Publication Year: 2013 , Page(s): 5566 - 5576
    Cited by:  Papers (3)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (3997 KB) |  | HTML iconHTML  

    We consider the problem of universal decoding for arbitrary, finite-alphabet unknown channels in the random coding regime. For a given random coding distribution and a given class of metric decoders, we propose a generic universal decoder whose average error probability is, within a subexponential multiplicative factor, no larger than that of the best decoder within this class of decoders. Since the optimum, maximum likelihood (ML) decoder of the underlying channel is not necessarily assumed to belong to the given class of decoders, this setting suggests a common generalized framework for: 1) mismatched decoding, 2) universal decoding for a given family of channels, and 3) universal coding and decoding for deterministic channels using the individual sequence approach. The proof of our universality result is fairly simple, and it is demonstrated how some earlier results on universal decoding are obtained as special cases. We also demonstrate how our method extends to more complicated scenarios, like incorporation of noiseless feedback, the multiple access channel, and continuous alphabet channels. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • The Entropy of Conditional Markov Trajectories

    Publication Year: 2013 , Page(s): 5577 - 5583
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1590 KB) |  | HTML iconHTML  

    To quantify the randomness of Markov trajectories with fixed initial and final states, Ekroot and Cover proposed a closed-form expression for the entropy of trajectories of an irreducible finite state Markov chain. Numerous applications, including the study of random walks on graphs, require the computation of the entropy of Markov trajectories conditional on a set of intermediate states. However, the expression of Ekroot and Cover does not allow for computing this quantity. In this paper, we propose a method to compute the entropy of conditional Markov trajectories through a transformation of the original Markov chain into a Markov chain that exhibits the desired conditional distribution of trajectories. Moreover, we express the entropy of Markov trajectories-a global quantity-as a linear combination of local entropies associated with the Markov chain states. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Local Pinsker Inequalities via Stein's Discrete Density Approach

    Publication Year: 2013 , Page(s): 5584 - 5591
    Cited by:  Papers (2)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (2391 KB) |  | HTML iconHTML  

    Pinsker's inequality states that the relative entropy between two random variables X and Y dominates the square of the total variation distance between X and Y. In this paper, we introduce generalized Fisher information distances and prove that these also dominate the square of the total variation distance. To this end, we introduce a general discrete Stein operator for which we prove a useful covariance identity. We illustrate our approach with several examples. Whenever competitor inequalities are available in the literature, the constants in ours are at least as good, and, in several cases, better. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Affine Moments of a Random Vector

    Publication Year: 2013 , Page(s): 5592 - 5599
    Cited by:  Papers (1)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1748 KB) |  | HTML iconHTML  

    An affine invariant pth moment measure is defined for a random vector and used to prove sharp moment-entropy inequalities that are more general and stronger than standard moment-entropy inequalities. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.

Aims & Scope

IEEE Transactions on Information Theory publishes papers concerned with the transmission, processing, and utilization of information.

Full Aims & Scope

Meet Our Editors

Editor-in-Chief
Frank R. Kschischang

Department of Electrical and Computer Engineering