By Topic

Information Theory, IEEE Transactions on

Issue 2 • Date March 1988

Filter Results

Displaying Results 1 - 25 of 27
  • Capacity and cutoff rate calculations for a concatenated coding system

    Page(s): 212 - 222
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1090 KB)  

    A model is developed for a concatenated coding system, and from it the overall channel cutoff rate and capacity is found using random coding arguments. The effects of interleaving between the inner and outer codes and of the availability of side information to the outer decoder are considered. The performance of several specific concatenated coding systems is calculated and used for comparison with the random coding result. From the analysis, a number of general conclusions regarding the design of concatenated coding systems are presented. All the results are derived assuming a block inner code.<> View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • On the existence of optimum cyclic burst correcting codes over GF( q)

    Page(s): 329 - 332
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (440 KB)  

    A cyclic b-burst correcting code over GF(q) of redundancy r and length n=(qr-b+1-1)/(q-1) is said to be optimum. It is proved that a necessary condition for the existence of such a code is the existence of a square-free polynomial in GF(q)[x] of degree b-1 which is not divisible by x such that its period and the degrees of its irreducible factors are relatively prime to q-1. Moreover, if such a polynomial exists, then there are an infinite number of optimum cyclic b-burst correcting codes over GF(q) View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Finding parity in a simple broadcast network

    Page(s): 176 - 180
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (404 KB)  

    A broadcast network of N+1 nodes is considered in which each binary digit transmitted by each node is received by every other node via a binary symmetric channel of given transition probability. The errors on these channels are independent over transmitters, receivers and time. Each node has a binary state, and the problem is to construct a distributed algorithm to find the parity of the set of states with some given reliability. It is shown that this can be done with O(ln(ln N)) bits of communication from each node. Communicating all the node states to one node can be accomplished with only marginally more communication View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A large automorphism group decreases the number of computations in the construction of an optimal encoder/decoder pair for linear block code

    Page(s): 333 - 338
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (600 KB)  

    For a linear block code it is shown how the number of computations needed for the determination of an optimal encoder/decoder pair can be reduced by using the code's automorphism group. Furthermore, it is shown that the use of an unequal-error-protection-optical generator matrix and a minimum-weight coset leader decoder is suboptimal for a q-ary symmetry channel, but their determination needs less computational effort View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • The capacity of the arbitrarily varying channel revisited: positivity, constraints

    Page(s): 181 - 193
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (960 KB)  

    A well-known result of R. Ahlswede (1970) asserts that the deterministic code capacity of an arbitrarily varying channel (AVC), under the average-error-probability criterion, either equals its random code capacity or else is zero. A necessary and sufficient condition is identified for deciding between these alternative, namely, the capacity is zero if and only if the AVC is symmetrizable. The capacity of the AVC is determined with constraints on the transmitted codewords as well as on the channel state sequences, and it is demonstrated that it may be positive but less than the corresponding random code capacity. A special case of the results resolves a weakened version of a fundamental problem of coding theory View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A general class of lower bounds in parameter estimation

    Page(s): 338 - 342
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (440 KB)  

    A general class of Bayesian lower bounds on moments of the error in parameter estimation is formulated, and it is shown that the Cramer-Rao, the Bhattacharyya, the Bobrovsky-Zakai, and the Weiss-Weinstein lower bounds are special cases in the class. The bounds can be applied to the estimation of vector parameters and any given function of the parameters. The extension of these bounds to multiple parameter is discussed View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A new geometric capacity characterization of a discrete memoryless channel

    Page(s): 318 - 321
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (304 KB)  

    A novel geometrical characterization of the capacity of a discrete memoryless channel is proposed. It is based on Csiszar's theorem, which represents the capacity using the Kullback-Leibler discrimination information. A new geometrical capacity computing method is obtained View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Reliable computation by formulas in the presence of noise

    Page(s): 194 - 197
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (324 KB)  

    It is shown that if formulas are used to compute Boolean functions in the presence of randomly occurring failures then: (1) there is a limit strictly less than 1/2 to the failure probability per gate that can be tolerated, and (2) formulas that tolerate failures must be deeper (and, therefore, compute more slowly) than those that do not. The heart of the proof is an information-theoretic argument that deals with computation and errors in very general terms. The strength of this argument is that it applies with equal ease no matter what types of gate are available. Its weaknesses is that it does not seem to predict quantitatively the limiting value of the failure probability or the ratio by which computation proceeds more slowly in the presence of failures View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • On classification with empirically observed statistics and universal data compression

    Page(s): 278 - 286
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (520 KB)  

    Classification with empirically observed statistics is studied for finite alphabet sources. Efficient universal discriminant functions are described and shown to be related to universal data compression. It is demonstrated that if one of the probability measure of the two classes is not known, it is still possible to define a universal discrimination function which performs as the optimal (likelihood ratio) discriminant function (which can be evaluated only if the probability measures of the two classes are available). If both of the probability measures are not available but training vectors from at least one of the two classes are available, it is demonstrated that no discriminant function can perform efficiency of the length of the training vectors does not grow at least linearly with the length of the classified vector. A universal discriminant function is introduced and shown to perform efficiently when the length of the training vectors grows linearly with the length of the classified sequence, in the sense that it yields an error exponent that is arbitrarily close to that of the optimal discriminant function View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A generalized two-threshold detection procedure

    Page(s): 347 - 352
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (424 KB)  

    A modified sequential procedure for testing binary hypotheses with different means, proposed by C.C. Lee and J.B. Thomas (ibid., vol.IT-30, no.1, p.16-23, Jan. 1984), is generalized for application to the case of multiple hypotheses with different means/variances of the Gaussian distribution. The method constitutes a two-threshold test for fixed-size packages of samples with a sequential procedure of discarding the package for which no decision is reached and subsequently testing a new package. The objective is to find an optimum package size N0 which leads to the minimum overall average sample number (ASN) for a given overall error probability. An optimization algorithm is developed to extend the application of the Lee-Thomas procedure to the M-ary case. Performance characteristics of the generalized two-threshold (GTT) test procedure are compared with those of conventional sequential as well as fixed-sample-size (FSS) methods. It is shown for the M-ary different means/variances cases that for low error rates the number of samples required by the GTT test is, on the average, approximately half that needed by a FSS test. However, it is somewhat more than the ASN obtained with a conventional sequential test. With decreasing error probabilities the GTT test performance approaches that of conventional sequential methods View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Optimal linear-quadratic systems for detection and estimation

    Page(s): 304 - 311
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (688 KB)  

    The problem of linear-quadratic systems for detection has long been solved by assuming the deflection criterion and Gaussian noise. It is shown here that the Gaussian assumption can be removed, and a complete solution is presented for an arbitrary probability distribution with finite fourth-order moments. The optimal solution can always be obtained by solving a linear system of equations. Some properties of the optimal systems are developed for particular examples of nonGaussian noise. It is shown that there is a strong relationship between linear-quadratic optimal detection and optimal estimation, which extends results known for the purely linear case View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Estimation via compressed information

    Page(s): 198 - 211
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (892 KB)  

    Some results from classical estimation theory are extended to the case in which data must be communicated from several places where observations are made to the place where the estimate is generated. Particular emphasis is placed on determining how the variance of an unbiased estimator depends on the communication rates. Explicit result are given for Gaussian sources View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Poisson models and mean-squared error for correlator estimator of time delay

    Page(s): 287 - 303
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1260 KB)  

    A method for modeling large errors in correlation-based time-delay estimation is developed in terms of level-crossing probabilities. The level-crossing interpretation for peak ambiguity leads directly to an exact expression for the probability of large error involving the hazard function associated with the level-crossing process. Two models for the distribution of the error over the level-crossing time yield approximations to the mean-square error (MSE) that involve the low-order (<4) finite-dimensional distributions of the associated level-crossing process. Application of an inhomogeneous Poisson model for the level crossings reduces the form of the approximations to a weighted sum of the Cramer-Rao lower bound and the second moment of a function of the level-crossing intensity over time. Explicit expressions for the large error probability and the MSE approximations are obtained under a Gaussian model for the correlator statistics. Results of computer simulation are presented that indicate the accuracy of the approximations View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Relations between Belini-Tartara, Chazan-Zakai-Ziv, and Wax-Ziv lower bounds

    Page(s): 342 - 343
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (184 KB)  

    A class of lower bounds on the mean-square error in parameter estimation is presented that are based on the Belini-Tartara lower bound (1974). The Wax-Ziv lower bound (1977) is shown to be a special case in the class. These bounds often are significantly tighter than the Chazan-Zakai-Ziv lower bound (1975) when the parameter to be estimated is subject to ambiguity and threshold effects View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Random access communication and graph entropy

    Page(s): 312 - 314
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (232 KB)  

    A probabilistic problem that arises for conflict resolution in random-access communication is treated. An earlier conjecture is disproved and a technique for finding lower bounds on the number of graphs of given structure needed to cover all edges of a given graph is developed View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Spread ambiguity functions

    Page(s): 343 - 347
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (460 KB)  

    A generalized spread ambiguity function is defined, and its relationship to P.M. Woodward's (1953) ambiguity function for different models is developed. A derivation and discussion of the properties of the spread ambiguity function are presented. It is shown that smoothing the ambiguity function by the scattering function of the channel does not alter the properties of the ambiguity function. Measures of RMS correlation time and RMS correlation bandwidth of nonstationary processes are defined. An example is given to demonstrate how spread ambiguity functions can be used to help in signal design when dealing with doubly spread targets View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Lower bounds for binary covering codes

    Page(s): 326 - 329
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (340 KB)  

    G.D. Chen et al. (ibid., vol.IT-32, p.680-94, 1986) presented two new lower bounds for K(n,R), where K(n,R) denotes the minimum cardinality of a binary code of length n and covering radius R. The author shows that a slight modification gives further improvements and some examples are given to confirm the argument. Codes that have a certain partitioning property are considered View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Sequential decoding for multiple access channels

    Page(s): 246 - 259
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1024 KB)  

    The use of sequential decoding in multiple access channels is considered. The Fano metric, which achieves all achievable rates in the one-user case, fails to do so in the multiuser case. A new metric is introduced and an inner bound is given to it achievable rate region. This inner bound region is large enough to encourage the use of sequential decoding in practice. The new metric is optimal, in the sense of achieving all achievable rates, in the case of one-user and pairwise-reversible channels. Whether the metric is optimal for all multiple access channels remains an open problem. It is noted that even in the one-user case, the new metric differs from the Fano metric in a nontrivial way, showing that the Fano metric is not uniquely optimal for such channels. A new and stricter criterion of achievability in sequential decoding is also introduced and examined View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Combined coding and modulation: theory and applications

    Page(s): 223 - 236
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1028 KB)  

    The theoretical aspects of the encoding process are investigated, resulting in a precise definition of linear codes together with theorems that clarify how they can be obtained. A particular subset of linear codes, called superlinear codes, for which the performance analysis is highly simplified is identified. The most relevant performance measures for the analysis of this class of codes are discussed. The minimum Euclidean distance and the event and bit error probabilities are found analytically using the uniform error property (when applicable) or variations on it. This yields accurate upper and lower bounds to the error rate at the price of reasonable computational complexity. The theory is then applied to the search for `good' codes and to their performance evaluation. The cases of 16- and 32-PSK codes, which are good candidates for use in digital satellite transmission, are considered. Several new results in terms of error event and bit error probabilities are presented, showing considerable gains in terms of SNR with respect to the uncoded case View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Almost asymptotically optimal flag encoding of the integers

    Page(s): 324 - 326
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (272 KB)  

    A simple prefix-free encoding scheme for the positive integers is proposed in which a flag of f zeros is used to mark the end of a codeword, and bit stuffing is used to prevent premature appearance of the flag in the conventional binary coding of the integers. It is shown that this coding scheme is universal in the sense defined by P. Elias (1975) and that for large f its asymptotic efficiency is virtually 1 View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Fixed-length strategies for the binary multiplying channel

    Page(s): 314 - 318
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (436 KB)  

    An achievable rate region for the binary multiplying channel is established such that block codes exist at all rate pairs within this region, with error probability that decreases exponentially with increasing block length. The result are generalized to asymmetrical rate pair View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Bounds on the extreme eigenvalues of positive-definite Toeplitz matrices

    Page(s): 352 - 355
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (368 KB)  

    Easily computable bounds on the extreme eigenvalues of positive semidefinite (PSD) Toeplitz matrices are presented. The bounds are especially suitable for matrices of relatively small dimension. The bounds are derived for the wider class of PSD Hermitian matrices and interpreted via the Levinson-Durbin Algorithm for Toeplitz matrices. As a by-product of this derivation an order-recursive algorithm for the eigenvector/eigenvalue decomposition is obtained, and certain properties of the eigenvalues distribution are revealed View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Improved sphere bounds on the covering radius of codes

    Page(s): 237 - 245
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (540 KB)  

    The sphere bound is a trivial lower bound on K(n,R), the minimal cardinality of any binary code of length n and with covering radius R. By simple arguments it is considerably improved, to K(n,1)⩾2 n/n for n even. A table of lower and upper bounds on K(n,R) for n⩽33, R ⩽10 is included View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A decomposition approach to CPM

    Page(s): 260 - 270
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (744 KB)  

    It is shown that any continuous-phase-modulation (CPM) system can be decomposed into a continuous-phase encoder and a memoryless modulator in such a way that the former is a linear (modulo some integer P) time-invariant sequential circuit and the latter is also time invariant. This decomposition is exploited to obtain alternative realizations of the continuous-phase encoder (and hence of CPM) and also to obtain alternative forms of the optimum decoding algorithm. When P is a prime p so that the encoder is linear over the finite field GF(p), it is shown that cascading it with an outside convolutional encoder is equivalent to a single convolutional encoder. It is pointed out that the cascade of the modulator, the waveform channel (which it is assumed is characterized by additive white Gaussian noise), and the demodulator that operates over one symbol interval yield a discrete memoryless channel that can be studied without the distractions introduced by continuous-phase encoding View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • On the capacity of channels with block memory

    Page(s): 322 - 324
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (292 KB)  

    The capacity of channels with block memory is investigated. It is shown that, when the problem is modeled as a game-theoretic problem, the optimum coding and noise distributions when block memory is permitted are independent from symbol to symbol within a block. Optimal jamming strategies are also independent from symbol to symbol within a block View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.

Aims & Scope

IEEE Transactions on Information Theory publishes papers concerned with the transmission, processing, and utilization of information.

Full Aims & Scope

Meet Our Editors

Editor-in-Chief
Frank R. Kschischang

Department of Electrical and Computer Engineering