IEEE Transactions on Information Theory

Volume 37 Issue 6 • Nov. 1991

Filter Results

Displaying Results 1 - 25 of 33
  • Information theoretic inequalities

    Publication Year: 1991, Page(s):1501 - 1518
    Cited by:  Papers (308)  |  Patents (1)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (1552 KB)

    The role of inequalities in information theory is reviewed, and the relationship of these inequalities to inequalities in other branches of mathematics is developed. The simple inequalities for differential entropy are applied to the standard multivariate normal to furnish new and simpler proofs of the major determinant inequalities in classical mathematics. The authors discuss differential entrop... View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • On identification via multiway channels with feedback

    Publication Year: 1991, Page(s):1519 - 1526
    Cited by:  Papers (15)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (766 KB)

    For the case of complete feedback, a fairly unified theory of identification is presented. Its guiding principle is the discovery that communicators (sender and receiver) must set up a common random experiment with maximal entropy and use it as randomization for a suitable identification technique. It is shown how this can be done in a constructive manner. The proof of optimality (weak converse) i... View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Information rates for a discrete-time Gaussian channel with intersymbol interference and stationary inputs

    Publication Year: 1991, Page(s):1527 - 1539
    Cited by:  Papers (97)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (1297 KB)

    Bounds are presented on I/sub i.i.d./-the achievable information rate for a discrete Gaussian Channel with intersymbol interference (ISI) present and i.i.d. channel input symbols governed by an arbitrary predetermined distribution p/sub x/(x). The lower and upper bounds on I/sub i.i.d./ and I are formulated. The bounds on I/sub i.i.d./ are calculated for independent equiprobably binary channel sym... View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • On the capacity of a direct-detection photon channel with intertransition-constrained binary input

    Publication Year: 1991, Page(s):1540 - 1550
    Cited by:  Papers (16)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (837 KB)

    The classical directed detection photon channel is modeled by an output nu /sub t/ (observed signal) describing the photon-arrival Poisson (count) process with intensity (rate) lambda /sub t/+ lambda /sub 0/, where lambda /sub t/ (photons/s) is the channel input (information carrying) intensity and lambda /sub 0/ (photons/s) is the dark current intensity. Upper and lower bounds on the capacity of ... View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Trellis-coded vector quantization

    Publication Year: 1991, Page(s):1551 - 1566
    Cited by:  Papers (68)  |  Patents (3)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (1531 KB)

    Trellis-coded quantization is generalized to allow a vector reproduction alphabet. Three encoding structures are described, several encoder design rules are presented, and two design algorithms are developed. It is shown that for a stationary ergodic vector source, if the optimized trellis-coded vector quantization reproduction process is jointly stationary and ergodic with the source, then the qu... View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Projective Reed-Muller codes

    Publication Year: 1991, Page(s):1567 - 1576
    Cited by:  Papers (49)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (745 KB)

    A class of codes in the Reed-Muller family, the projective Reed-Muller codes (PRM codes), is studied. The author defines the PRM codes of all orders and discusses their relation to polynomial codes. The exact parameters of PRM codes are given. The duals are characterized, and, in parallel to the classical works on generalized Reed-Muller codes, the cyclic properties are studied. Tables over parame... View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Runlength-limited codes for mixed-error channels

    Publication Year: 1991, Page(s):1577 - 1585
    Cited by:  Papers (21)  |  Patents (3)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (729 KB)

    The mixed-error channel (MC) combines the binary symmetric channel and the peak shift channel. The construction of (d, k) constrained t-MC-error-correcting block codes is described. It is demonstrated that these codes can achieve a code rate close to the (d, k) capacity. The encoding and decoding procedures are described. The performance of the construction depends on a particular partitioning of ... View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Universal classification for hidden Markov models

    Publication Year: 1991, Page(s):1586 - 1594
    Cited by:  Papers (6)  |  Patents (2)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (850 KB)

    Binary hypotheses testing using empirically observed statistics is studied in the Neyman-Pearson formulation for the hidden Markov model (HMM). An asymptotically optimal decision rule is proposed and compared to the generalized likelihood ratio test (GLRT), which has been shown in earlier studies to be asymptotically optimal for simpler parametric families. The proof of the main theorem is provide... View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • How much information can one bit of memory retain about a Bernoulli sequence?

    Publication Year: 1991, Page(s):1595 - 1604
    Cited by:  Papers (5)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (735 KB)

    The maximin problem of the maximization of the minimum amount of information that a single bit of memory retains about the entire past is investigated. The problem is to estimate the supremum over all possible sequences of update rules of the minimum information that the bit of memory at epoch (n+1) retains about the previous n inputs. Using only elementary techniques, it is shown that the maximin... View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Cutting and stacking: a method for constructing stationary processes

    Publication Year: 1991, Page(s):1605 - 1617
    Cited by:  Papers (9)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (1264 KB)

    Cutting and stacking, a general method for constructing examples of stationary, ergodic, finite-alphabet processes with desired properties, is described. It has been used to construct counterexamples in ergodic theory. Recently, it has also been used to construct examples of interest in information theory and probability theory. The method builds a stationary ergodic process by describing sample p... View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A theoretical analysis of Monte Carlo algorithms for the simulation of Gibbs random field images

    Publication Year: 1991, Page(s):1618 - 1628
    Cited by:  Papers (6)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (975 KB)

    Various theoretical and computational issues about the algorithms' behavior are addressed. The concept of relative entropy is introduced as the primary analytical tool, and convergence of the simulation algorithms is discussed in terms of the relative entropy. This approach allows a view of the simulation of Gibbs random field images as a constrained, convex optimization problem, and it results in... View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Deterministic properties of analog median filters

    Publication Year: 1991, Page(s):1629 - 1640
    Cited by:  Papers (2)  |  Patents (12)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (1156 KB)

    Two key deterministic properties of analog median filters are established. The first is that root signals are characterized by a special class of locally monotone functions, and the second is that the repeated application of a median filter produces a root signal which retains significant features in the original data. Both results are extensions of well known facts for discrete median filters. In... View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • The entropy of a randomly stopped sequence

    Publication Year: 1991, Page(s):1641 - 1644
    Cited by:  Papers (9)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (328 KB)

    A Wald-like equation is proved for the entropy of a randomly stopped sequence of independent identically distributed discrete random variables X/sub 1/, X/sub 2/. . ., with a nonanticipating stopping time N. The authors first define a general stopping time and the associated stopped sequence, and then present the two main theorems for the entropy of a stopped sequence. The formal proofs of the lem... View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • The entropy theorem via coding bounds

    Publication Year: 1991, Page(s):1645 - 1647
    Cited by:  Papers (3)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (324 KB)

    The entropy theorem (also known as the Shannon-McMillan-Breiman theory or the asymptotic equipartition theorem) asserts that, for a stationary ergodic finite alphabet process, the sequence-(1/n)log p(x/sub 1//sup n/) converges almost surely to the entropy-rate H of the process. The entropy theorem has been used to establish asymptotic bounds on the performance of noiseless codes. Here, the coding ... View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Two proofs of Pinsker's conjecture concerning arbitrarily varying channels

    Publication Year: 1991, Page(s):1647 - 1649
    Cited by:  Papers (14)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (317 KB)

    M.S. Pinsker (1990) conjectured the following theorem: for an arbitrary varying channel (AVC), every rate below the random code capacity is achievable with deterministic list codes of constant list size, if the average error criterion is used. Two proofs of this theorem are given.<<ETX>> View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Sequence estimation and synchronization from nonsynchronized samples

    Publication Year: 1991
    Cited by:  Papers (25)  |  Patents (11)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (827 KB)

    The authors study the problem of maximum-likelihood sequence estimation and synchronization from samples of the output of a matched filter taken at integer multiples of the symbol rate, which is assumed perfectly known by the receiver. A general analysis is presented of sampled receivers that handle arbitrary baseband pulse shapes and arbitrary sampling rates. It is observed that the optimal proce... View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • The distribution functions of certain random geometric series concerning intersymbol interference

    Publication Year: 1991, Page(s):1657 - 1662
    Cited by:  Papers (6)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (565 KB)

    The author presents some numerical methods for the calculation of the distribution functions of certain random geometric series. The semicontraction mapping approach of A. Huzii and H. Sugiyama (Electron. Commun. Jap.., vol.53-A, p.21-30, 1970) is generalized to give a convergent solution for most cases of interest. Also, initial approximations are discussed based on the moments of the distributio... View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A note on difference methods for the prediction of band-limited signals from past samples

    Publication Year: 1991, Page(s):1662 - 1665
    Cited by:  Papers (1)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (271 KB)

    Recent results are extended on difference methods for the prediction of bandlimited signals, obtained by D.H. Mugler and W. Splettstosser (1986, 1987) and by D.H. Mugler (1990), are extended to generalized bandlimited signals defined by M. Zakai (1965) and A.J. Lee (1975, 1976). This makes the difference method applicable to a wider class of signals.<<ETX>> View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A counterexample to a Voronoi constellation conjecture

    Publication Year: 1991, Page(s):1665 - 1666
    Cited by:  Papers (1)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (184 KB)

    Voronoi constellations are of interest because it is possible to save on signal power by choosing a lattice for which the second moment of the Voronoi region is close to that of a sphere with the same dimensionality. A Voronoi constellation based on the lattice partition Lambda / Lambda /sub s/ is the set of points in some translate Lambda +a that fall within the Voronoi region R( Lambda /sub s/).... View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • On rates of convergence of general information criteria in signal processing when the noise covariance matrix is arbitrary

    Publication Year: 1991, Page(s):1667 - 1671
    Cited by:  Papers (6)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (394 KB)

    The authors give exponential-type bounds on the probabilities of detection error under certain conditions. The bounds tend to zero rapidly as the sample size increases. These procedures are referred to as general information criterion (GIC) procedures since each is consistent and, under certain conditions, the rate of convergence of the estimate of the number of signals to the true value is rapid.... View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Parameter estimator based on a minimum discrepancy criterion: a Bayesian approach

    Publication Year: 1991, Page(s):1671 - 1675
    Cited by:  Papers (2)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (513 KB)

    A new estimation criterion based on the discrepancy between the estimator's error covariance and its information lower bound is proposed. This discrepancy measure criterion tries to take the information content of the observed data into account. A minimum discrepancy estimator (MDE) is then obtained under a linearity assumption. This estimator is shown to be equivalent to the maximum likelihood es... View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Signal sets matched to groups

    Publication Year: 1991, Page(s):1675 - 1682
    Cited by:  Papers (87)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (941 KB)

    Recently, linear codes over Z/sub M/ (the ring of integers mod M) have been presented that are matched to M-ary phase modulation. The general problem of matching signal sets to generalized linear algebraic codes is addressed based on these codes. A definition is given for the notion of matching. It is shown that any signal set in N-dimensional Euclidean space that is matched to an abstract group i... View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Recursive estimation in mixture models with Markov regime

    Publication Year: 1991, Page(s):1683 - 1690
    Cited by:  Papers (25)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (702 KB)

    A recursive algorithm is proposed for estimation of parameters in mixture models, where the observations are governed by a hidden Markov chain. The often badly conditioned information matrix is estimated, and its inverse is incorporated into the algorithm. The performance of the algorithm is studied by simulations of a symmetric normal mixture. The algorithm seems to be stable and produce approxim... View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • The transient solution of time-dependent M/M/1 queues

    Publication Year: 1991, Page(s):1690 - 1696
    Cited by:  Papers (20)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (607 KB)

    The transient behavior of time-dependent M/M/1 queues is studied. The boundary probability function pi /sub 0/(t), which is the probability that the queue is empty at time t, is shown with analyticity arguments to satisfy a Volterra-type integral equation. The boundary integral equation is derived, and a numerical algorithm is used to solve the integral equation and to find the expected queue size... View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • New lower bounds for asymmetric and unidirectional codes

    Publication Year: 1991, Page(s):1696 - 1704
    Cited by:  Papers (13)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (882 KB)

    New lower bounds on the sizes of asymmetric codes and unidirectional codes are presented. Various methods are used, three of them of special interest. The first is a partitioning method that is a modification of a method used to construct constant weight codes. The second is a combining codes method that is used to obtain a new code from a few others. The third method is shortening by weights that... View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.

Aims & Scope

IEEE Transactions on Information Theory publishes papers concerned with the transmission, processing, and utilization of information.

Full Aims & Scope

Meet Our Editors

Editor-in-Chief
Alexander Barg

Department of Electrical and Computer Engineering and the Institute for Systems Research, University of Maryland

email: abarg-ittrans@ece.umd.edu