IEEE Transactions on Information Theory

Volume 47 Issue 5 • July 2001

Filter Results

Displaying Results 1 - 25 of 43
  • Information geometry on hierarchy of probability distributions

    Publication Year: 2001, Page(s):1701 - 1711
    Cited by:  Papers (159)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (339 KB) | HTML iconHTML

    An exponential family or mixture family of probability distributions has a natural hierarchical structure. This paper gives an "orthogonal" decomposition of such a system based on information geometry. A typical example is the decomposition of stochastic dependency among a number of random variables. In general, they have a complex structure of dependencies. Pairwise dependency is easily represent... View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Strong optimality of the normalized ML models as universal codes and information in data

    Publication Year: 2001, Page(s):1712 - 1717
    Cited by:  Papers (77)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (136 KB) | HTML iconHTML

    We show that the normalized maximum-likelihood (NML) distribution as a universal code for a parametric class of models is closest to the negative logarithm of the maximized likelihood in the mean code length distance, where the mean is taken with respect to the worst case model inside or outside the parametric class. We strengthen this result by showing that, when the data generating models are re... View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Multiple-description vector quantization with lattice codebooks: design and analysis

    Publication Year: 2001, Page(s):1718 - 1734
    Cited by:  Papers (132)  |  Patents (4)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (500 KB) | HTML iconHTML

    The problem of designing a multiple-description vector quantizer with lattice codebook /spl Lambda/ is considered. A general solution is given to a labeling problem which plays a crucial role in the design of such quantizers. Numerical performance results are obtained for quantizers based on the lattices A/sub 2/ and Z/sup i/, i=1, 2, 4, 8, that make use of this labeling algorithm. The high-rate s... View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • On the robustness of single-loop sigma-delta modulation

    Publication Year: 2001, Page(s):1735 - 1744
    Cited by:  Papers (19)  |  Patents (2)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (273 KB) | HTML iconHTML

    Sigma-delta modulation, a widely used method of analog-to-digital (A/D) signal conversion, is known to be robust to hardware imperfections, i.e., bit streams generated by slightly imprecise hardware components can be decoded comparably well. We formulate a model for robustness and give a rigorous analysis for single-loop sigma-delta modulation applied to constant signals (DC inputs) for N time cyc... View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • On unique, multiset, and set decipherability of three-word codes

    Publication Year: 2001, Page(s):1745 - 1757
    Cited by:  Papers (3)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (430 KB) | HTML iconHTML

    The concepts of unique decipherability (UD), multiset decipherability (MSD), and set decipherability (SD) of codes were developed to handle some special problems in the transmission of information. In unique decipherability, different sequences of codewords carry different information. In multiset decipherability, the information of interest is the multiset of codewords used in the encoding proces... View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • On coset weight distributions of the Z/sub 4/-linear Goethals codes

    Publication Year: 2001, Page(s):1758 - 1772
    Cited by:  Papers (8)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (448 KB) | HTML iconHTML

    We study the coset weight distributions of two well-known families of codes: the three-error-correcting binary Z/sub 4/-linear Goethals codes of length N=2/sup m+1/, m/spl ges/3 odd, and the Z/sub 4/-linear Goethals codes over Z/sub 4/ of length n=N/2=2/sup m/. The hard case is the weight distributions of cosets of weight 4. To know the weight distribution of the coset of weight 4 we have to know ... View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Binary images of cyclic codes over Z/sub 4/

    Publication Year: 2001, Page(s):1773 - 1779
    Cited by:  Papers (27)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (205 KB) | HTML iconHTML

    We determine all linear cyclic codes over Z/sub 4/ of odd length whose Gray images are linear codes (or, equivalently, whose Nechaev-Gray (1989) image are linear cyclic codes or are linear cyclic codes). View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Constructing critical indecomposable codes

    Publication Year: 2001, Page(s):1780 - 1795
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (420 KB) | HTML iconHTML

    Critical indecomposable codes were introduced by Assmus (see ibid., vol.44, p.612-29, 1998), who also gave a recursive construction for these objects. One of the key ingredients in the construction is an auxiliary code, which is an indecomposable code of minimum distance at least 3. In terms of actually being able to construct all critical indecomposable codes, however, Assmus leaves many unanswer... View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Lower bounds on the anticipation of encoders for input-constrained channels

    Publication Year: 2001, Page(s):1796 - 1812
    Cited by:  Papers (1)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (470 KB) | HTML iconHTML

    An input-constrained channel S is defined as the set of words generated by a finite labeled directed graph. It is shown that every finite-state encoder with finite anticipation (i.e., with finite decoding delay) for S can be obtained through state-splitting rounds applied to some deterministic graph presentation of S, followed by a reduction of equivalent states. Furthermore, each splitting round ... View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Stream-oriented turbo codes

    Publication Year: 2001, Page(s):1813 - 1831
    Cited by:  Papers (14)  |  Patents (2)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (501 KB) | HTML iconHTML

    This work considers the design and performance of a stream-oriented approach to turbo codes which avoids the need for data framing. The stream paradigm applies to both serial and parallel turbo codes using continuous, free-running constituent encoders along with continuous, periodic interleavers. A stream-oriented turbo code based on parallel concatenated convolutional codes (PCCC) is considered a... View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • MAP symbol decoding in channels with error bursts

    Publication Year: 2001, Page(s):1832 - 1838
    Cited by:  Papers (4)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (175 KB) | HTML iconHTML

    We analyze the forward-backward algorithm for the maximum a posteriori (MAP) decoding of information transmitted over channels with memory and propose a suboptimal forward-only algorithm. We assume that both the information source and channel are described using hidden Markov models (HMMs). The algorithm lends itself to parallel implementation and pipelining. We apply the algorithm to MAP decoding... View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Partial-prefix synchronizable codes

    Publication Year: 2001, Page(s):1839 - 1848
    Cited by:  Papers (2)  |  Patents (2)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (287 KB) | HTML iconHTML

    A new class of codes for frame synchronization is proposed. Commonly, the beginning of every fixed or variable-length frame is identified by a given contiguous sequence called a prefix. To avoid the occurrence of the prefix elsewhere in the frame, a prefix synchronizable code (PS-code) is used. PS-codes have the property that the prefix does not occur in any codeword or in any concatenation of cod... View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Twofold universal prediction schemes for achieving the finite-state predictability of a noisy individual binary sequence

    Publication Year: 2001, Page(s):1849 - 1866
    Cited by:  Papers (11)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (542 KB) | HTML iconHTML

    The problem of predicting the next outcome of an individual binary sequence corrupted by noise using finite memory, is considered. The conditional finite-state (FS) predictability of an infinite individual sequence given its noisy version is defined as the minimum fraction of errors that can be made by any FS predictor fed by the noisy version. It is proved that the conditional FS predictability c... View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Optimization of Barron density estimates

    Publication Year: 2001, Page(s):1867 - 1883
    Cited by:  Papers (5)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (556 KB) | HTML iconHTML

    We investigate a nonparametric estimator of the probability density introduced by Barron (1988, 1989). Earlier papers established its consistency in a strong sense, e.g., in the expected information divergence or expected chi-square divergence. This paper pays main attention to the expected chi-square divergence criterion. We give a new motivation of the Barron estimator by showing that a maximum-... View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Adaptive context trees and text clustering

    Publication Year: 2001, Page(s):1884 - 1901
    Cited by:  Papers (8)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (568 KB) | HTML iconHTML

    In the finite-alphabet context we propose four alternatives to fixed-order Markov models to estimate a conditional distribution. They consist in working with a large class of variable-length Markov models represented by context trees, and building an estimator of the conditional distribution with a risk of the same order as the risk of the best estimator for every model simultaneously, in a condit... View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Rademacher penalties and structural risk minimization

    Publication Year: 2001, Page(s):1902 - 1914
    Cited by:  Papers (97)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (421 KB) | HTML iconHTML

    We suggest a penalty function to be used in various problems of structural risk minimization. This penalty is data dependent and is based on the sup-norm of the so-called Rademacher process indexed by the underlying class of functions (sets). The standard complexity penalties, used in learning problems and based on the VC-dimensions of the classes, are conservative upper bounds (in a probabilistic... View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Pattern recognition algorithms based on space-filling curves and orthogonal expansions

    Publication Year: 2001, Page(s):1915 - 1927
    Cited by:  Papers (10)  |  Patents (3)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (365 KB) | HTML iconHTML

    The classical problem of constructing a multidimensional pattern classifier in the Bayesian framework is considered. Preprocessing of the learning sequence by a quasi-inverse of a space-filling curve is proposed and properties of space-filling curves which are necessary to obtain Bayes risk consistency are indicated. The learning sequence transformed into the unit interval is used to estimate the ... View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Performance of reduced-rank linear interference suppression

    Publication Year: 2001, Page(s):1928 - 1946
    Cited by:  Papers (159)  |  Patents (1)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (492 KB) | HTML iconHTML

    The performance of reduced-rank linear filtering is studied for the suppression of multiple-access interference. A reduced-rank filter resides in a lower dimensional space, relative to the full-rank filter, which enables faster convergence and tracking. We evaluate the large system output signal-to-interference plus noise ratio (SINR) as a function of filter rank D for the multistage Wiener filter... View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Cumulant-based independence measures for linear mixtures

    Publication Year: 2001, Page(s):1947 - 1956
    Cited by:  Papers (16)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (264 KB) | HTML iconHTML

    This paper deals with independence measures for linear mixtures of mutually independent random variables. Such measures, also known as contrasts, constitute useful criteria in solving blind source separation problems. By making use of the Schur convexity properties, we show that it is possible to define a wide-ranging class of contrast functions based on the auto-cumulants of the components of the... View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • On the performance of linear parallel interference cancellation

    Publication Year: 2001, Page(s):1957 - 1970
    Cited by:  Papers (35)  |  Patents (3)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (374 KB) | HTML iconHTML

    This paper analyzes the performance of the linear parallel interference cancellation (LPIC) multiuser detector in a synchronous multiuser communication scenario with binary signaling, nonorthogonal multiple access interference, and an additive white Gaussian noise channel. The LPIC detector has been considered in the literature lately due to its low computational complexity, potential for good per... View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • The throughput of hybrid-ARQ protocols for the Gaussian collision channel

    Publication Year: 2001, Page(s):1971 - 1988
    Cited by:  Papers (429)  |  Patents (2)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (527 KB) | HTML iconHTML

    In next-generation wireless communication systems, packet-oriented data transmission will be implemented in addition to standard mobile telephony. We take an information-theoretic view of some simple protocols for reliable packet communication based on "hybrid-ARQ," over a slotted multiple-access Gaussian channel with fading and study their throughput (total bit per second per hertz) and average d... View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Iterative construction of optimum signature sequence sets in synchronous CDMA systems

    Publication Year: 2001, Page(s):1989 - 1998
    Cited by:  Papers (204)  |  Patents (3)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (234 KB) | HTML iconHTML

    Optimum signature sequence sets that maximize the capacity of single-cell synchronous code division multiple access (CDMA) systems have been identified. Optimum signature sequences minimize the total squared correlation (TSC); they form a set of orthogonal sequences, if the number of users is less than or equal to the processing gain, and a set of Welch (1994) bound equality (WBE) sequences, other... View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Error exponents for the two-user Poisson multiple-access channel

    Publication Year: 2001, Page(s):1999 - 2016
    Cited by:  Papers (13)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (556 KB) | HTML iconHTML

    The error exponent of the two-user Poisson multiple-access channel under peak and average power constraints, but unlimited in bandwidth, is considered. First, a random coding lower bound on the error exponent is obtained, and an extension of Wyner's (1988) single-user codes is shown to be exponentially optimum for this case as well. Second, the sphere-packing bounding technique suggested by Burnas... View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Gauges and gauge transformations for uncertainty description of geometric structure with indeterminacy

    Publication Year: 2001, Page(s):2017 - 2028
    Cited by:  Papers (17)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (301 KB) | HTML iconHTML

    This paper presents a consistent theory for describing indeterminacy and uncertainty of three-dimensional (3-D) reconstruction from a sequence of images. First, we give a group-theoretical analysis of gauges and gauge transformations. We then discuss how to evaluate the reliability of the solution that has indeterminacy and extend the Cramer-Rao lower bound to incorporate internal indeterminacy. W... View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • On the whiteness of high-resolution quantization errors

    Publication Year: 2001, Page(s):2029 - 2038
    Cited by:  Papers (21)  |  Patents (1)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (424 KB) | HTML iconHTML

    A common belief in quantization theory says that the quantization noise process resulting from uniform scalar quantization of a correlated discrete-time process tends to be white in the limit of small distortion ("high resolution"). A rule of thumb for this property to hold is that the source samples have a "smooth" joint distribution. We give a precise statement of this property, and generalize i... View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.

Aims & Scope

IEEE Transactions on Information Theory publishes papers concerned with the transmission, processing, and utilization of information.

Full Aims & Scope

Meet Our Editors

Editor-in-Chief
Alexander Barg

Department of Electrical and Computer Engineering and the Institute for Systems Research, University of Maryland

email: abarg-ittrans@ece.umd.edu