By Topic

Information Theory, IEEE Transactions on

Issue 3 • Date March 2008

Filter Results

Displaying Results 1 - 25 of 43
  • Table of contents

    Page(s): C1 - C4
    Save to Project icon | Request Permissions | PDF file iconPDF (48 KB)  
    Freely Available from IEEE
  • IEEE Transactions on Information Theory publication information

    Page(s): C2
    Save to Project icon | Request Permissions | PDF file iconPDF (38 KB)  
    Freely Available from IEEE
  • Error Exponents for Variable-Length Block Codes With Feedback and Cost Constraints

    Page(s): 945 - 963
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (499 KB) |  | HTML iconHTML  

    Variable-length block-coding schemes are investigated for discrete memoryless channels with ideal feedback under cost constraints. Upper and lower bounds are found for the minimum achievable probability of decoding error Pe,min as a function of constraints R, P, and tau on the transmission rate, average cost, and average block length, respectively. For given R and P, the lower and upper bounds to the exponent -( ln Pe,min )/tau are asymptotically equal as tau rarr infin. The resulting reliability function,limtaurarrinfin(-In Pe,min)/tau as a function of R and V, is concave in the pair (R,P) and generalizes the linear reliability function of Burnashev to include cost constraints. The results are generalized to a class of discrete-time memoryless channels with arbitrary alphabets, including additive Gaussian noise channels with amplitude and power constraints. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Lautum Information

    Page(s): 964 - 975
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (339 KB) |  | HTML iconHTML  

    A popular way to measure the degree of dependence between two random objects is by their mutual information, defined as the divergence between the joint and product-of-marginal distributions. We investigate an alternative measure of dependence: the lautum information defined as the divergence between the product-of-marginal and joint distributions, i.e., swapping the arguments in the definition of mutual information. Some operational characterizations and properties are provided for this alternative measure of information. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Multiple-Access Channels With Confidential Messages

    Page(s): 976 - 1002
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1074 KB) |  | HTML iconHTML  

    A discrete memoryless multiple-access channel (MAC) with confidential messages is studied, where two users attempt to transmit common information to a destination and each user also has private (confidential) information intended for the destination. This channel generalizes the classical MAC model in that each user also receives channel outputs, and hence may obtain the confidential information sent by the other user from the channel output it receives. However, each user views the other user as a wiretapper or eavesdropper, and wishes to keep its confidential information as secret as possible from the other user. The level of secrecy of the confidential information is measured by the equivocation rate, i.e., the entropy rate of the confidential information conditioned on channel outputs at the wiretapper (the other user). The performance measure is the rate-equivocation tuple that includes the common rate, two private rates, and two equivocation rates as components. The set that includes all achievable rate-equivocation tuples is referred to as the capacity-equivocation region. The case of perfect secrecy is particularly of interest, in which each user's confidential information is perfectly hidden from the other user. The set that includes all achievable rates with perfect secrecy is referred to as the secrecy capacity region. For the MAC with two confidential messages, in which both users have confidential messages for the destination, inner bounds on the capacity-equivocation region, and secrecy capacity region are obtained. It is demonstrated that there is a tradeoff between the two equivocation rates (secrecy levels) achieved for the two confidential messages. For the MAC with one confidential message, in which only one user (user 1) has private (confidential) information for the destination, inner and outer bounds on the capacity-equivocation region are derived. These bounds match partially, and hence the capacity-equivocation region is partially characteri- - zed. Furthermore, the outer bound provides a tight converse for the case of perfect secrecy, and hence establishes the secrecy capacity region. A class of degraded MACs with one confidential message is further studied, and the capacity-equivocation region and the secrecy capacity region are established. These results are further explored via two example channels: the binary and Gaussian MACs. For both channels, the capacity-equivocation regions and the secrecy capacity regions are obtained. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • An Algebraic, Analytic, and Algorithmic Investigation on the Capacity and Capacity-Achieving Input Probability Distributions of Finite-Input– Finite-Output Discrete Memoryless Channels

    Page(s): 1003 - 1023
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (477 KB) |  | HTML iconHTML  

    In this paper, we investigate the capacity and capacity-achieving input probability distributions (IPDs) of finite-input-finite-output discrete memoryless channels (DMCs). In the general respect, we establish a novel and simple characterization for the capacity-achieving IPDs of a DMC, which is equivalent to the conventional Kuhn-Tucker conditions. We then prove a conjecture of Majani and Rumsey, which claims that every probability component of each capacity-achieving IPD of a DMC with positive capacity is less than 1-e-1, where e = 2.71828182... is the base of natural logarithms. It remains an open problem whether there exists an explicit closed-form solution for the capacity and capacity-achieving IPDs of a general finite-input-finite-output DMC, except for the two-input-two-output DMC. In the algebraic respect, we demonstrate that there does not, in general, exist an algebraic solution for the capacity-achieving IPDs of an m-input-n-output DMC for any m ges 2 . and any n ges 3. In the analytic respect, however, we can obtain an explicit closed-form analytic solution, represented as an infinite series, for the capacity-achieving IPD of a two-input-three-output DMC. We also provide a formula for the average capacity of weakly symmetric DMCs and show that the average capacity in nats per channel use of the n-input-m-output weakly symmetric DMCs increases for n ges 2 but has a finite limit of 1 - gamma as n rarr infin, where gamma = 0.57721566... is Euler's constant. In the algorithmic respect, the convergence of the Arimoto-Blahut algorithm is proved in a direct and elementary way. A new and simple iterative algorithm for calculating a capacity-achieving IPD is then proposed, which is provably convergent for all DMCs with positive transition probabilities. Finally, the characterization and determination of the set of all capacity-achieving IPDs of a DMC are addressed. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Bounds for Codes in Products of Spaces, Grassmann, and Stiefel Manifolds

    Page(s): 1024 - 1035
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (305 KB) |  | HTML iconHTML  

    Upper bounds are derived for codes in Stiefel and Grassmann manifolds with given minimum chordal distance. They stem from upper bounds for codes in the product of unit spheres and projective spaces. The new bounds are asymptotically better than the previously known ones. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Soft-Output BEAST Decoding With Application to Product Codes

    Page(s): 1036 - 1049
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (766 KB) |  | HTML iconHTML  

    A bidirectional efficient algorithm for searching code trees (BEAST) is proposed for efficient soft-output decoding of block codes and concatenated block codes. BEAST operates on trees corresponding to the minimal trellis of a block code and finds a list of the most probable codewords. The complexity of the BEAST search is significantly lower than the complexity of trellis-based algorithms, such as the Viterbi algorithm and its list generalizations. The outputs of BEAST, a list of best codewords and their metrics, are used to obtain approximate a posteriori probabilities (APPs) of the transmitted symbols, yielding a soft-input soft-output (SISO) symbol decoder referred to as the BEAST-APP decoder. This decoder is employed as a component decoder in iterative schemes for decoding of product and incomplete product codes. Its performance and convergence behavior are investigated using extrinsic information transfer (EXIT) charts and compared to existing decoding schemes. It is shown that the BEAST-APP decoder achieves performances close to the Bahl-Cocke-Jelinek-Raviv (BCJR) decoder with a substantially lower computational complexity. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • On the Hardness of Decoding the Gale–Berlekamp Code

    Page(s): 1050 - 1060
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (301 KB) |  | HTML iconHTML  

    The Gale-Berlekamp (in short, GB) code is the dual code of the binary product code in which the horizontal and vertical constituent codes are both the parity code. It is shown that the problem of deciding whether there is a codeword of the GB code within a prescribed distance from a given received word, is NP-complete. The problem remains hard (in a well-defined sense) even if the decoder is allowed unlimited preprocessing that depends only on the code length. While the intractability of maximum-likelihood decoding (MLD) for specific codes has already been shown by Bruck and Naor, Lobstein, and Guruswami and Vardy, the result herein seems to be the first that shows hardness for a "natural" code (in particular, without any tailoring of the definition or the parameters of the code to suit the hardness proof). In contrast, it is also shown that, with respect to any memoryless binary-symmetric channel (BSC) with crossover probability less than 1/2, MLD can be implemented in linear time for all error events except for a portion that occurs with vanishing probability. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A New Linear Programming Approach to Decoding Linear Block Codes

    Page(s): 1061 - 1072
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (472 KB) |  | HTML iconHTML  

    In this paper, we propose a new linear programming formulation for the decoding of general linear block codes. Different from the original formulation given by Feldman, the number of total variables to characterize a parity-check constraint in our formulation is less than twice the degree of the corresponding check node. The equivalence between our new formulation and the original formulation is proven. The new formulation facilitates to characterize the structure of linear block codes, and leads to new decoding algorithms. In particular, we show that any fundamental polytope is simply the intersection of a group of the so-called minimum polytopes, and this simplified formulation allows us to formulate the problem of calculating the minimum Hamming distance of any linear block code as a simple linear integer programming problem with much less auxiliary variables. We then propose a branch-and-bound method to compute a lower bound to the minimum distance of any linear code by solving a corresponding linear integer programming problem. In addition, we prove that, for the family of single parity-check (SPC) product codes, the fractional distance and the pseudodistance are both equal to the minimum distance. Finally, we propose an efficient algorithm for decoding SPC product codes with low complexity and maximum-likelihood (ML) decoding performance. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A Quasi-Random Approach to Space–Time Codes

    Page(s): 1073 - 1085
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (427 KB) |  | HTML iconHTML  

    This paper presents a quasi-random approach to space-time (ST) codes. The basic principle is to transmit randomly interleaved versions of forward error correction (FEC)-coded sequences simultaneously from all antennas in a multilayer structure. This is conceptually simple, yet still very effective. It is also flexible regarding the transmission rate, antenna numbers, and channel conditions (e.g., with intersymbol interference). It provides a unified solution to various applications where the traditional ST codes may encounter difficulties. We outline turbo-type iterative joint detection and equalization algorithms with complexity (per FEC-coded bit) growing linearly with the transmit antenna number and independently of the layer number. We develop a signal-to-noise-ratio (SNR) evolution technique and a bounding technique to assess the performance of the proposed code in fixed and quasi-static fading channels, respectively. These performance assessment techniques are very simple and reasonably accurate. Using these techniques as a searching tool, efficient power allocation strategies are examined, which can greatly enhance the system performance. Simulation results show that the proposed code can achieve near-capacity performance with both low and high rates at low decoding complexity. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • On the User Selection for MIMO Broadcast Channels

    Page(s): 1086 - 1107
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1239 KB) |  | HTML iconHTML  

    In this paper, a downlink communication system, in which a base station (BS) equipped with antennas communicates with users each equipped with receive antennas, is considered. An efficient suboptimum algorithm is proposed for selecting a set of users in order to maximize the sum-rate throughput of the system, in a Rayleigh-fading environment. For the asymptotic case when tends to infinity, the necessary and sufficient conditions in order to achieve the maximum sum-rate throughput, such that the difference between the achievable sum-rate and the maximum value approaches zero, is derived. The complexity of our algorithm is investigated in terms of the required amount of feedback from the users to the BS, as well as the number of searches required for selecting the users. It is shown that the proposed method is capable of achieving a large portion of the sum-rate capacity, with a very low complexity. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Quantization Bounds on Grassmann Manifolds and Applications to MIMO Communications

    Page(s): 1108 - 1123
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (609 KB) |  | HTML iconHTML  

    The Grassmann manifold Gn,p (L) is the set of all p-dimensional planes (through the origin) in the n-dimensional Euclidean space Ln, where L is either R or C. This paper considers the quantization problem in which a source in Gn,p (L) is quantized through a code in Gn,q (L), with p and q not necessarily the same. The analysis is based on the volume of a metric ball in Gn,p (L) with center in Gn,q (L), and our chief result is a closed-form expression for the volume of a metric ball of radius at most one. This volume formula holds for arbitrary n, p, q, and L, while previous results pertained only to some special cases. Based on this volume formula, several bounds are derived for the rate-distortion tradeoff assuming that the quantization rate is sufficiently high. The lower and upper bounds on the distortion rate function are asymptotically identical, and therefore precisely quantify the asymptotic rate-distortion tradeoff. We also show that random codes are asymptotically optimal in the sense that they achieve the minimum possible distortion in probability as n and the code rate approach infinity linearly. Finally, as an application of the derived results to communication theory, we quantify the effect of beamforming matrix selection in multiple-antenna communication systems with finite rate channel state feedback. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Broadcast in MIMO Systems Based on a Generalized QR Decomposition: Signaling and Performance Analysis

    Page(s): 1124 - 1138
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1033 KB) |  | HTML iconHTML  

    A simple signaling method for broadcast channels with multiple-transmit multiple-receive antennas is proposed. In this method, for each user, the direction in which the user has the maximum gain is determined. The best user in terms of the largest gain is selected. The corresponding direction is used as the modulation vector (MV) for the data stream transmitted to the selected user. The algorithm proceeds in a recursive manner where in each step, the search for the best direction is performed in the null space of the previously selected MVs. It is demonstrated that with the proposed method, each selected MV has no interference on the previously selected MVs. Dirty-paper coding is used to cancel the remaining interference. For the case that each receiver has one antenna, the presented scheme coincides with the known scheme based on Gram-Schmidt orthogonalization (QR decomposition). To analyze the performance of the scheme, an upper bound on the cumulative distribution function (CDF) of each subchannel is derived which is used to establish the diversity order and the asymptotic sum-rate of the scheme. It is shown that using fixed rate codebooks, the diversity order of the jth data stream, 1 les j les M, is equal to N(M - j + 1)(K - j + 1), where M, N, and K indicate the number of transmit antennas, the number of receive antennas, and the number of users, respectively. Furthermore, it is proven that the throughput of this scheme scales as M log log(K) and asymptotically (K rarr infin) tends to the sum-capacity of the multiple-input multiple-output (MIMO) broadcast channel. The simulation results indicate that the achieved sum-rate is close to the sum-capacity of the underlying broadcast channel. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • On the Derivation of the Exact, Closed-Form Capacity Formulas for Receiver-Sided Correlated MIMO Channels

    Page(s): 1139 - 1161
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (792 KB) |  | HTML iconHTML  

    This paper solves the problem of finding a closed-form expression for the average information-theoretic capacity of wireless systems with an arbitrary number of transmitter and receiver antennas. It is assumed that only the receivers have (perfect) knowledge of the channel state and that fading correlation is receiver-sided. The main purpose of the paper is accomplished by introducing a few very simple concepts and performing some clear-cut algebraic manipulations, making the presentation virtually self-contained. The results show a substantial capacity reduction in the presence of correlation between receiver antennas. It is also shown that employing linear arrays with nonuniform spacings may improve the system capacity, and a simple technique to exploit this possibility is presented. Isotropic and nonisotropic propagation scenarios are studied. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Analysis and Optimization of Diagonally Layered Lattice Schemes for MIMO Fading Channels

    Page(s): 1162 - 1185
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (822 KB) |  | HTML iconHTML  

    Embodiments of the diagonal Bell Laboratories layered space-time (D-BLAST) architecture for multiple-input-multiple-output (MIMO) communication are developed wherein information symbol vectors are encoded using codewords from a lattice code [called a diagonally layered lattice (DLL) code], which are formatted onto the diagonals of a space-time frame. Decoding is done using a sphere decoder for each diagonal based on soft statistics obtained after zero forcing (ZF) or minimum-mean-square-error (MMSE) filtering and decision feedback. These operations give rise to an effective parallel channel model with channel gains with nonidentical statistics and additive noise which is Gaussian in the ZF-filtering case and non-Gaussian in the MMSE-filtering case. The so-called full modulation diversity (FMD) property is nevertheless shown to yield the maximum achievable diversity orders over the MIMO channel for both the ZF- and the MMSE-filtering-based decoders respectively, for any arbitrary fading distribution. In the case of the independent, identically distributed (i.i.d.) Rayleigh fading MIMO channel with K-transmit and N-receive antennas (with NgesK), these diversity orders are NK-K(K-1)/2 and NK for ZF- and MMSE-filtering-based decoding, respectively. The error probability analysis also yields a design criterion for optimizing transmit power allocations. Several lattice design methods are proposed for the effective parallel channel models. Two methods are proposed to achieve high coding gain in the Rayleigh fading MIMO channel; a third method is proposed that minimizes the exact symbol error probability (SEP) and can be tailored for any given fading distribution. A novel soft decision feedback decoder is also proposed based on the list sphere decoder to mitigate error propagation due to hard decision feedback. The salient feature of the proposed DLL schemes is that they have nearly full rate and full (or high) diversi- - ty order and yet a much lower decoding complexity than other existing full rate, full diversity space-time block codes (STBCs). The frame error probability (FEP) performance of the optimized DLL schemes for moderate-to-high spectral efficiencies and a wide range of signal-to-noise ratios (SNRs) can be quite close to the performance of the best performing, but more complex to decode, STBCs. Moreover, the proposed DLL schemes significantly outperform other existing MIMO systems of comparable decoding complexity. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Diversity–Multiplexing Tradeoff and Outage Performance for Rician MIMO Channels

    Page(s): 1186 - 1196
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (721 KB) |  | HTML iconHTML  

    In this paper, we analyze the diversity-multiplexing tradeoff (DMT), originally introduced by Zheng and Tse, and outage performance for Rician multiple-input-multiple-output (MIMO) channels. The DMT characteristics of Rayleigh and Rician channels are shown to be identical. In a high signal-to-noise ratio (SNR) regime, the log-log plot of outage probability versus SNR curve for a Rician channel is a shifted version of that for the corresponding Rayleigh channel. The SNR gap between the outage curves of the Rayleigh and Rician channels is derived. The DMT and outage performance are also analyzed for Rician multiple-input-single-output (MISO)/single-input-multiple-output (SIMO) channels over a finite SNR regime. A closed-form expression for the outage probability is derived and the finite SNR DMT characteristic is analyzed. It is observed that the maximum diversity gain can be achieved at some finite SNR-the maximum gain tends to increase linearly with the Rician factor. The finite SNR diversity gain is shown to be a linear function of the finite SNR multiplexing gain. The consistency between the DMTs for finite and infinite SNRs is also shown. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • User Cooperation in the Absence of Phase Information at the Transmitters

    Page(s): 1197 - 1206
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (420 KB) |  | HTML iconHTML  

    In this paper, a multiuser communication system in which wireless users cooperate to transmit information to a base station is considered. The proposed scheme can significantly enlarge the achievable rate region, provided that the wireless connections between pairs of cooperating users are stronger than the connection from every user to the base station. The gains in transmission rate remain substantial even when the channel phase information is only available at the receivers, not at the transmitters. In the proposed scheme, a transmission period is divided into two time intervals. During the first time interval, wireless users send data to the base station and to the neighboring users simultaneously using a broadcast channel paradigm. During the second time interval, the users cooperate to transmit information to the base station. The achievable rate region corresponding to this paradigm is characterized under a random phase channel model for a two-user system. Results are then generalized to a multiple-user scenario. For fixed system parameters, the achievable rate region is strictly larger than that of the traditional multiple-access channel, thereby allowing a fair distribution of the wireless resources among users. Numerical analysis suggests that cooperating with a single partner is enough to achieve most of the benefits associated with cooperation. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Incremental Redundancy Cooperative Coding for Wireless Networks: Cooperative Diversity, Coding, and Transmission Energy Gains

    Page(s): 1207 - 1224
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (668 KB) |  | HTML iconHTML  

    We study an incremental redundancy (IR) cooperative coding scheme for wireless networks. To exploit the distributed spatial diversity we propose a cluster-based collaborating strategy for a quasi-static Rayleigh-fading channel model. Our scheme allows for enhancing the reliability performance of a direct communication over a single hop. The collaborative cluster consists of M - 1 nodes between the sender and the destination. The transmitted message is encoded using a mother code which is partitioned into M blocks each assigned to one of M transmission slots. In the first slot, the sender broadcasts its information by transmitting the first block, and its helpers attempt to decode this message. In the remaining slots, each of the next M - 1 blocks is sent either through a helper which has successfully decoded the message or directly by the sender where a dynamic schedule is based on the ACK-based feedback from the cluster. By employing powerful good codes including turbo, low-density parity-check (LDPC), and repeat-accumulate (RA) codes, our approach illustrates the benefit of collaboration through not only a cooperation diversity gain but also a coding advantage. The basis of our error rate performance analysis is based on a derived code threshold for the Bhattacharyya distance which describes the behavior of good codes. The new simple code threshold is based on the modified Shulman-Feder bound and the relationship between the Bhattacharyya parameter and the channel capacity for an arbitrary binary-input symmetric-output memoryless channel. An average frame-error rate (FER) upper bound and its asymptotic (in signal-to-noise ratio (SNR)) version are derived as a function of the average fading channel SNRs and the code threshold. Based on the asymptotic bound, we investigate both the diversity, the coding, and the transmission energy gain in the high and moderate SNR regimes for three different scenarios: transmitter clustering, receiver clustering, and cluster hoppin- - g. We observe that the energy saving of the IR cooperative coding scheme is universal for all good code families in the sense that the gain does not depend on the sender-to-destination distance and the code threshold. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Power-Efficient Resource Allocation for Time-Division Multiple Access Over Fading Channels

    Page(s): 1225 - 1240
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1111 KB) |  | HTML iconHTML  

    We investigate resource allocation policies for time-division multiple access (TDMA) over fading channels in the power-limited regime. For frequency-flat block-fading channels and transmitters having full channel state information (CSI), we first minimize power under a weighted sum average rate constraint and show that the optimal rate and time allocation policies can be obtained by a greedy water-filling approach with linear complexity in the number of users. Subsequently, we pursue power minimization under individual average rate constraints and establish that the optimal resource allocation also amounts to a greedy water-filling solution. Our approaches not only provide fundamental power limits when each user can support an infinite-size capacity-achieving codebook (continuous rates), but also yield guidelines for practical designs where users can only support a finite set of adaptive modulation and coding modes (discrete rates). View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Max-Product for Maximum Weight Matching: Convergence, Correctness, and LP Duality

    Page(s): 1241 - 1251
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (370 KB) |  | HTML iconHTML  

    Max-product "belief propagation" (BP) is an iterative, message-passing algorithm for finding the maximum a posteriori (MAP) assignment of a discrete probability distribution specified by a graphical model. Despite the spectacular success of the algorithm in many application areas such as iterative decoding and combinatorial optimization, which involve graphs with many cycles, theoretical results about both the correctness and convergence of the algorithm are known in only a few cases (see section I for references). In this paper, we prove the correctness and convergence of max-product for finding the maximum weight matching (MWM) in bipartite graphs. Even though the underlying graph of the MWM problem has many cycles, somewhat surprisingly we show that the max-product algorithm converges to the correct MWM as long as the MWM is unique. We provide a bound on the number of iterations required and show that for a graph of size n, the computational cost of the algorithm scales as O(n3), which is the same as the computational cost of the best known algorithms for finding the MWM. We also provide an interesting relation between the dynamics of the max-product algorithm and the auction algorithm, which is a well-known distributed algorithm for solving the MWM problem. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Randomness Criteria in Terms of  {\alpha } -Divergences

    Page(s): 1252 - 1261
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (284 KB) |  | HTML iconHTML  

    Vovk's randomness criterion characterizes sequences that are random relative to two distinct computable probability measures. The uniqueness of the criterion lies in the fact that, unlike the standard criterion based on the likelihood ratio test, it is expressed in terms of a geometrical quantity, the Hellinger distance, on the space of probability measures. In this paper, we generalize the randomness criterion to a wider class of geometrical quantities, the -divergences with . The nonextendibility of the criterion across the boundaries is investigated in connection with the likelihood ratio test and information geometry. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Recursive Lower Bounds on the Nonlinearity Profile of Boolean Functions and Their Applications

    Page(s): 1262 - 1272
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (332 KB) |  | HTML iconHTML  

    The nonlinearity profile of a Boolean function (i.e., the sequence of its minimum Hamming distances nlr(f) to all functions of degrees at most r, for r ges 1) is a cryptographic criterion whose role against attacks on stream and block ciphers has been illustrated by many papers. It plays also a role in coding theory, since it is related to the covering radii of Reed-Muller codes. We introduce a method for lower-bounding its values and we deduce bounds on the second-order nonlinearity for several classes of cryptographic Boolean functions, including the Welch and the multiplicative inverse functions (used in the S-boxes of the Advanced Encryption Standard (AES)). In the case of this last infinite class of functions, we are able to bound the whole profile, and we do it in an efficient way when the number of variables is not too small. This allows showing the good behavior of this function with respect to this criterion as well. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Optimal Prefix Codes for Infinite Alphabets With Nonlinear Costs

    Page(s): 1273 - 1286
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (424 KB) |  | HTML iconHTML  

    Let P={p(i)} be a measure of strictly positive probabilities on the set of nonnegative integers. Although the countable number of inputs prevents usage of the Huffman algorithm, there are nontrivial P for which known methods find a source code that is optimal in the sense of minimizing expected codeword length. For some applications, however, a source code should instead minimize one of a family of nonlinear objective functions, beta-exponential means, those of the form loga Sigmaip(i)an(i), where n(i) is the length of the ith codeword and a is a positive constant. Applications of such minimizations include a novel problem of maximizing the chance of message receipt in single-shot communications (a<1) and a previously known problem of minimizing the chance of buffer overflow in a queueing system (a>1). This paper introduces methods for finding codes optimal for such exponential means. One method applies to geometric distributions, while another applies to distributions with lighter tails. The latter algorithm is applied to Poisson distributions and both are extended to alphabetic codes, as well as to minimizing maximum pointwise redundancy. The aforementioned application of minimizing the chance of buffer overflow is also considered. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Network Coding Capacity With a Constrained Number of Coding Nodes

    Page(s): 1287 - 1291
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (221 KB) |  | HTML iconHTML  

    We study network coding capacity under a constraint on the total number of network nodes that can perform coding. That is, only a certain number of network nodes can produce coded outputs, whereas the remaining nodes are limited to performing routing. We prove that every nonnegative, monotonically nondecreasing, eventually constant, rational-valued function on the nonnegative integers is equal to the capacity as a function of the number of allowable coding nodes of some directed acyclic network. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.

Aims & Scope

IEEE Transactions on Information Theory publishes papers concerned with the transmission, processing, and utilization of information.

Full Aims & Scope

Meet Our Editors

Editor-in-Chief
Frank R. Kschischang

Department of Electrical and Computer Engineering