By Topic

Information Theory, IEEE Transactions on

Issue 5 • Date Sept. 1992

Filter Results

Displaying Results 1 - 24 of 24
  • Comments on "Evaluating distance spectra and performance bounds of trellis codes on channels with intersymbol interference

    Page(s): 1623 - 1625
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (305 KB)  

    It is shown that the method presented by C. Schlegel (see ibid., vol.37, no.3, p.627-34, May 1991) for evaluating the distance spectra of trellis codes on channels with intersymbol interference is incorrect. A counterexample is presented along with comments on the feasibility of correcting the method.<> View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Worst-case power-constrained noise for binary-input channels

    Page(s): 1494 - 1511
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (2691 KB)  

    Additive noise channels with binary-valued inputs and real-valued outputs are considered. The maximum error probability and the minimum channel capacity achieved by any power-constrained noise distribution are obtained. A general framework which applies to a variety of performance measures shows that the least-favorable noise distribution is, in general, a mixture of two lattice probability mass functions. The framework holds for m-ary input constellations on finite-dimensional lattices. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Variable-length-to-variable length source coding: a greedy step-by-step algorithm

    Page(s): 1609 - 1617
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (676 KB)  

    Some results on variable-length-to-variable-length source coding are presented. A sufficient criterion for the asymptotic optimality of the generic V-V code is given. This also allows the study of the Tunstall-Huffman scheme performance. Then, a nonoptimal greedy approach to prefix codes is described, based on the informational divergence pseudometric and on the Gallager algorithm minimization of the current rate, associated with the subsequent extensions of the source probability distribution. Although optimality is not always reached, this technique can be usefully employed to improve the Tunstall-Huffman concatenation View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • New lower bounds for binary codes of asymmetric distance two

    Page(s): 1592 - 1597
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (392 KB)  

    Lower bounds for asymmetric single-error-correcting codes are derived. The codes are constructed by puncturing constant weight codes and by using a random coding argument View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Switched scalar quantizers for hidden Markov sources

    Page(s): 1455 - 1473
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1460 KB)  

    An algorithm for designing switched scalar quantizers for hidden Markov sources is described. The design problem is cast as a nonlinear optimization problem. The optimization variables are the thresholds and reproduction levels for each quantizer and the parameters defining the next-quantizer map. The cost function is the average distribution incurred by the system in steady-state. The next-quantizer map is treated as a stochastic map so that all of the optimization variables are continuous-valued, allowing the use of a gradient-based optimization procedure. This approach solves a major problem in the design of switched scalar quantizing systems, namely, that of determining an optimal next-quantizer decision rule. Details are given for computing the cost function and its gradient for weighted-squared-error distortion. Simulation results which compare the new system to current systems show that the present system performs better. It is observed that the optimal system can in fact have a next-quantizer map with stochastic components View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Minimum distance of logarithmic and fractional partial m-sequences

    Page(s): 1474 - 1482
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (588 KB)  

    Two results are presented concerning the partial periods (p-p's) of an m-sequence of period 2n-1. The first proves the existence of an m-sequence whose p-p's of length approximately (n+d log2 n) have minimum distance between d and 2d for small d. The second result is of an asymptotic nature and proves that the normalized minimum distance of p-p's whose length is any fraction of the period of the m-sequence, approaches 1/2 as the period of m-sequence tends to infinity View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A new class of bit- and byte-error control codes

    Page(s): 1617 - 1623
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (608 KB)  

    Error-control codes for byte-oriented systems are presented. The proposed codes are intended for systems wherein the erroneous bits tend to be confined to a small number of bytes. Mathematical techniques are developed for the construction of codes that can detect and correct such errors. Among the various codes presented, the codes for detection and correction of errors confined to a single byte are of particular interest. A decoding algorithm for these codes is also presented View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Nonparametric identification of Wiener systems

    Page(s): 1487 - 1493
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (540 KB)  

    A Wiener system, i.e., a system in which a linear dynamic part is followed by a nonlinear and memoryless one, is identified. No parametric restriction is imposed on the functional form of the nonlinear characteristic of the memoryless subsystem, and a nonparametric algorithm recovering the characteristic from input-output observations of the whole system is proposed. Its consistency is shown and the rate of convergence is given. An idea for identification of the impulse response of the linear subsystem is proposed. Results of numerical simulation are also presented View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Optimum codes of dimension 3 and 4 over GF(4)

    Page(s): 1564 - 1567
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (388 KB)  

    nq(k,d), the length of a q-ary optimum code for given k and d, for q=4 and k=3, 4 is discussed. The problem is completely solved for k=3, and the exact value of n4(4,d) is determined for all but 52 values of d View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Stability of linear predictors and numerical range of shift operators in normal spaces

    Page(s): 1483 - 1486
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (252 KB)  

    The zeros of predictor polynomials are shown to belong to the numerical range of a shift operator associated with the particular prediction problem under consideration. The numerical range consists of the classical field of values of the shift operator when the setting is Hilbert space, but a new definition is necessary when the setting is a general normed space. It is shown that a predictor polynomial is not stable in general. Nevertheless, for predictor polynomials in l p spaces, it is shown that their zeros belong to the open circular disk with radius 2 View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Zero-redundancy coding for unequal code symbol costs

    Page(s): 1583 - 1586
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (380 KB)  

    Source distributions that can be encoded with zero redundancy for the case of unequal code symbol costs are examined. These distributions provide a natural generalization of the binary, equal costs case for which these distributions are the dyadic distributions. These zero redundancy codes have the property that the expected proportion of codeword symbols given by a particular letter is equal to an experimental function of the code letter cost. The converse is not true in general; however, partial converse results hold. Maximum-entropy zero-redundancy distributions are easily identified through their connection with unequal cost coding for uniform sources View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Recursively indexed quantization of memoryless sources

    Page(s): 1602 - 1609
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (664 KB)  

    A recursively indexed scalar quantizer that performs as well as high-dimensional vector quantizers for several important sources, without the attendant complexity, is presented. The recursively indexed quantizer provides a simple technique, both in terms of design and operation, for use with entropy coding. It also provides a simple quantization technique for use in noisy channel conditions where a variable length code would be inappropriate but performance greater than that provided by Lloyd-Max quantizers is desired View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • On the main conjecture on geometric MDS codes

    Page(s): 1573 - 1577
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (376 KB)  

    A new way to attack the main conjecture on MDS codes for geometric codes is proposed. In particular, the conjecture for codes arising from curves of genus one or two when the cardinal of the ground field is large enough is proven View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A direct geometrical method for bounding the error exponent for any specific family of channel codes. I. Cutoff rate lower bound for block codes

    Page(s): 1548 - 1559
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (932 KB)  

    A direct, general, and conceptually simple geometrical method for determining lower and upper bounds on the error exponent of any specific family of channel block codes is presented. It is considered that a specific family of codes is characterized by a unique distance distribution exponent. The tight linear lower bound of slope -1 on the code family error exponent represents the code family cutoff rate bound. It is always a minimum of a sum of three functions. The intrinsic asymptotic properties of channel block codes are revealed by analyzing these functions and their relationships. It is shown that the random coding technique for lower-bounding the channel error exponent is a special case of this general method. The requirements that a code family should meet in order to have a positive error exponent and at best attain the channel error exponent are stated in a clear way using the (direct) distance distribution method presented View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Distribution estimation consistent in total variation and in two types of information divergence

    Page(s): 1437 - 1454
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1192 KB)  

    The problem of the nonparametric estimation of a probability distribution is considered from three viewpoints: the consistency in total variation, the consistency in information divergence, and consistency in reversed-order information divergence. These types of consistencies are relatively strong criteria of convergence, and a probability distribution cannot be consistently estimated in either type of convergence without any restrictions on the class of probability distributions allowed. Histogram-based estimators of distribution are presented which, under certain conditions, converge in total variation, in information divergence, and in reversed-order information divergence to the unknown probability distribution. Some a priori information about the true probability distribution is assumed in each case. As the concept of consistency in information divergence is stronger than that of convergence in total variation, additional assumptions are imposed in the cases of informational divergences View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A recurrence theorem for dependent processes with applications to data compression

    Page(s): 1561 - 1564
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (460 KB)  

    In an earlier work, Wyner and Ziv (see ibid., vol.35, no.6, p.1250-8, 1989) proved theorems on recurrence times for strings in a random sequence, and applied these theorems to data compression and the Lempel-Ziv algorithm. It is shown that one of these theorems holds under an essentially weaker hypothesis. The new proof is considerably simpler than the original View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A relation between Kolmogorov-Prokhorov's condition and Ohya's fractal dimensions

    Page(s): 1567 - 1570
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (324 KB)  

    A computational method giving upper bounds of ∈-entropy of stochastic processes satisfying Kolmogorov-Prokhorov's condition is studied, in terms of Shannon's entropy of the probability measures on C[0,1] induced by these processes. Moreover, this method and Ohya's fractal dimensions are applied to investigating the asymptotic behavior of ∈-entropy of such processes View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Algebraic decoding of the Zetterberg codes

    Page(s): 1570 - 1573
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (268 KB)  

    The Zetterberg codes are one of the best known families of double-error correcting binary linear codes. Unfortunately, no satisfactory decoding algorithm has been known for them until recently when an algebraic decoding algorithm was described by P. Kallquist (1989). It requires, however, to solve a quadratic equation in order to decide whether 2 or 3 errors have occurred. A simple criterion is derived to determine whether 1, 2, or 3 errors have occurred when a Zetterberg code is used for data transmission. Based on criterion a new decoding algorithm is proposed which is faster than the known one View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Effect of binary modulation codes with rate R=1/N on equivalent discrete-time models for channels with intersymbol interference

    Page(s): 1587 - 1592
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (416 KB)  

    For a data transmission system with intersymbol interference and noise in which signaling occurs by means of nonoverlapping rectangular pulses, a relation between the equivalent discrete-time models for uncoded and coded transmission is derived. It applies to binary modulation codes with rate R=1/N, where N is a positive integer. Examples suggest that these models are often affected by coding in a manner that is incompatible with a commonly adopted definition of coding gain View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Average-case interactive communication

    Page(s): 1534 - 1547
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1056 KB)  

    X and Y are random variables. Person Px knows X, Person Py knows Y, and both know the joint probability distribution of the pair (X,Y). Using a predetermined protocol, they communicate over a binary error-free channel in order for Py to learn X. Px may or may not learn Y. It is determined how many information bits must be transmitted (by both persons) on the average. The results show that, when the arithmetic average number of bits is considered, there is no asymptotic advantage to Px knowing Y in advance and four messages are asymptotically optimum. By contrast, for the worst-case number of bits, communication can be significantly reduced if Px knows Y in advance, and it is not known whether a constant number of messages is asymptotically optimum View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • When is the generalized likelihood ratio test optimal?

    Page(s): 1597 - 1602
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (524 KB)  

    The generalized likelihood ratio test (GLRT), which is commonly used in composite hypothesis testing problems, is investigated. Conditions for asymptotic optimality of the GLRT in the Neyman-Pearson sense are studied and discussed. First, a general necessary and sufficient condition is established, and then based on this, a sufficient condition, which is easier to verify, is derived. A counterexample where the GLRT is not optimal, is provided as well. A conjecture is stated concerning the optimality of the GLRT for the class of finite-state sources View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Sigma-delta modulation with leaky integration and constant input

    Page(s): 1512 - 1533
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1056 KB)  

    Exact descriptions of the behavior of quantization noise for single-loop, multistage (cascade), and multiloop sigma-delta modulators for a variety of input signals have been found during recent years under the assumption of ideal integration. The techniques used to solve the ideal integrator case do not easily extend to the more realistic model of a leaky integrator sigma-delta. In this paper a dynamical system representation for the leaky integrator sigma-delta with a constant input is developed. Several properties of the resulting piecewise monotone and piecewise linear transformation T on the interval [0,1] are derived and applied to analyze the behavior of the corresponding sigma-delta modulator. It is shown that almost all constant inputs yield a periodic output sequence and an asymptotically periodic quantizer error sequence. These sequences are described. It is also found that unlike the ideal system, inputs of different value contained in a certain range can produce identical output sequences View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Optimal and robust kernel algorithms for passive stochastic approximation

    Page(s): 1577 - 1583
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (532 KB)  

    The problem of estimating a root of an equation f(x )=0 is considered in the situation where the values of f( x) are measured with random errors at random points and the choice of these points cannot be controlled. Nonlinear modification of the recursive Hardle-Nixdorf method is studied. Almost sure and mean square convergence is proved, and the rate of convergence is estimated. The optimal choice of parameters and of a kernel is presented; it is shown that for the optimal procedure the lower bound for the accuracy of arbitrary methods of solving the problem is attained View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • On minimal decoding sets for the extended binary Golay code

    Page(s): 1560 - 1561
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (204 KB)  

    Minimal decoding sets consisting of fourteen permutations have been found for the (24,12,8) extended binary Golay code. It is shown that by properly sequencing the permutations of one such set, the average number of permutations required to decode a random received word can be minimized View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.

Aims & Scope

IEEE Transactions on Information Theory publishes papers concerned with the transmission, processing, and utilization of information.

Full Aims & Scope

Meet Our Editors

Editor-in-Chief
Frank R. Kschischang

Department of Electrical and Computer Engineering