IEEE Transactions on Information Theory

Volume 20 Issue 5 • September 1974

Filter Results

Displaying Results 1 - 25 of 33
  • Chernoff bounds on the error probability for the detection of non-Gaussian signals

    Publication Year: 1974, Page(s):569 - 577
    Cited by:  Papers (7)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (1435 KB)

    Chernoff bounds on the error probability for the detection of non-Gaussian stochastic signals in additive white Gaussian noise are computed. By the use of Fokker-Planck (F-P) equations and a certain conditional expectation, the quasi-transition function, an equation for time evolution of the Chernoff bound is obtained. This time evolution equation is solved exactly to give all previously known res... View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Fourier series and estimation on the circle with applications to synchronous communication--I: Analysis

    Publication Year: 1974, Page(s):577 - 583
    Cited by:  Papers (27)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (1032 KB)

    A wide variety of continuous- and discrete-time estimation problems on the circle<tex>S^1</tex>are considered with the aid of Fourier series analysis. Measurement and diffusion update equations are derived for the conditional expectation of certain functions of the parameter to be estimated, and we investigate the use of Fourier series to obtain easily implemented optimal estimation eq... View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Fourier series and estimation on the circle with applications to synchronous communication--II: Implementation

    Publication Year: 1974, Page(s):584 - 590
    Cited by:  Papers (20)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (998 KB)

    The practical implementation of the infinite-dimensional optimal estimation results presented in Part I of this series is considered. Several techniques are described in detail. Included among these is the so-called "assumed density" approximation technique. Finite-dimensional suboptimal filtering equations based on this method are derived for several of the phase-tracking/demodulation problems st... View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Single tone parameter estimation from discrete-time observations

    Publication Year: 1974, Page(s):591 - 598
    Cited by:  Papers (1230)  |  Patents (11)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (1039 KB)

    Estimation of the parameters of a single-frequency complex tone from a finite number of noisy discrete-time observations is discussed. The appropriate Cramér-Rao bounds and maximum-likelihood (MI.) estimation algorithms are derived. Some properties of the ML estimators are proved. The relationship of ML estimation to the discrete Fourier transform is exploited to obtain practical algorithms. The t... View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Correlation of a signal with an amplitude-distorted form of itself

    Publication Year: 1974, Page(s):598 - 601
    Cited by:  Papers (3)  |  Patents (2)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (507 KB)

    In comparing a signal<tex>f(t)</tex>with its amplitude-distorted form<tex>g(f(t))</tex>, where<tex>g(\cdot)</tex>is a monotonically increasing function of its argument, one is led to consider the correlation function \begin{equation} R(s) \triangleq \int_{-\infty}^{\infty} dtg (f(t))f(t-s). \end{equation} A rigorous proof is given of the inequality<tex>R(s... View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Optimal on-off signaling over linear time-varying stochastic channels

    Publication Year: 1974, Page(s):602 - 609
    Cited by:  Papers (4)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (1034 KB)

    The problems of optimum performance and signal design for on-off signaling over linear, time-varying stochastic channels are considered. The channel is modeled as a nonstationary linear time-varying Gaussian filter with additive white Gaussian noise at its output. A new expression for the probability of error is presented and is minimized under the constraint of fixed received energy. An approxima... View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Entropy inequalities for discrete channels

    Publication Year: 1974, Page(s):610 - 616
    Cited by:  Papers (29)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (928 KB)

    The sharp lower bound<tex>f(x)</tex>on the per-symbol output entropy for a given per-symbol input entropy<tex>x</tex>is determined for stationary discrete memoryless channels; it is the lower convex envelope of the bound<tex>g(x)</tex>for a single channel use. The bounds agree for all noiseless channels and all binary channels. However, for nonbinary channels,&l... View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Regular jump processes and their information processing

    Publication Year: 1974, Page(s):617 - 624
    Cited by:  Papers (8)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (1253 KB)

    A class of regular jump processes (RJP's) is introduced. An RJP is described in terms of the intensity function of its associated stochastic point process and the state-transition density of its embedded random-state sequence. Expressions for the joint occurrence statistics of these processes are derived. Assuming that an information stochastic process causally modulates an observed RJP, we obtain... View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • The ergodic decomposition of stationary discrete random processes

    Publication Year: 1974, Page(s):625 - 636
    Cited by:  Papers (43)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (1824 KB)

    The ergodic decomposition is discussed, and a version focusing on the structure of individual sample functions of stationary processes is proved for the special case of discrete-time random processes with discrete alphabets. The result is stronger in this case than the usual theorem, and the proof is both intuitive and simple. Estimation-theoretic and information-theoretic interpretations are deve... View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Majority decoding of some classes of binary cyclic codes

    Publication Year: 1974, Page(s):637 - 643
    Cited by:  Papers (1)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (1065 KB)

    This paper presents majority-decoding algorithms for four classes of binary cyclic codes. The classes are those for which the parity-check polynomial is: 1) the product of two primitive polynomials with relatively prime exponents and 2) the product of<tex>(x^r + 1)/(x + 1)</tex>and a primitive polynomial, where<tex>r \geq 3</tex>is odd and the exponents are relatively prime... View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Difference-preserving codes

    Publication Year: 1974, Page(s):643 - 649
    Cited by:  Papers (9)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (1272 KB)

    A code of integers into binary sequences is called a difference-preserving code (DP code) if it has the following two properties: 1) if the absolute value of the difference between two integers is less than or equal to a certain threshold, the Hamming distance between their codewords is equal to this value and 2) if the absolute value of the difference between two integers exceeds the threshold, t... View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • The cepstrum as a generalized function (Corresp.)

    Publication Year: 1974, Page(s):650 - 653
    Cited by:  Papers (2)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (615 KB)

    This correspondence investigates the possibility of defining the power or energy cepstrum as a generalized function. By applying this definition, the cepstrum can formally be computed, or a formal approximation can be achieved. The asymptotic behavior of the cepstrum is also examined. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Zipf's law and entropy (Corresp.)

    Publication Year: 1974, Page(s): 650
    Cited by:  Papers (7)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (138 KB)

    The estimate of the entropy of a language by assuming that the word probabilities follow Zipf's law is discussed briefly. Previous numerical results [3] on the vocabulary size implied by Zipf's law and entropy per word are corrected. The vocabulary size should be 12 366 words (not 8727 words) and the entropy per word 9.27 bits (not 11.82). View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Approximation of finite-energy random processes by continuous processes (Corresp.)

    Publication Year: 1974, Page(s):653 - 654
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (297 KB)

    It is shown that any random process with finite energy can be approximated arbitrarily closely by a mean-square continuous process. We obtain approximants that in addition have continuous sample paths with probability 1. By a straightforward extension of these results, the approximating process can be made to have differentiable sample paths. The approximation can be performed in real time, in the... View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • On optimal linear estimation of signals with general spectral distribution (Corresp.)

    Publication Year: 1974, Page(s):654 - 658
    Cited by:  Papers (3)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (813 KB)

    It is standard practice to assume that second-order stationary signal and noise processes involved in a linear estimation procedure have spectral densities. A heuristic justification may be based on the reasoning that the part of the signal having singular spectral distribution can be precisely determined, and the part of the noise having singular spectral distribution can be completely eliminated... View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Simplified relationships for the probability distribution of the phase of a sine wave in narrow-band normal noise (Corresp.)

    Publication Year: 1974, Page(s):658 - 661
    Cited by:  Papers (29)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (553 KB)

    A new form of the probability distribution of the phase of a sine wave in narrow-band normal noise is derived. It leads to several different simple expressions for the probability distribution which render the previously required numerical integration of the probability density function unnecessary. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A parametric procedure for imperfectly supervised learning with unknown class probabilities (Corresp.)

    Publication Year: 1974, Page(s):661 - 663
    Cited by:  Papers (2)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (386 KB)

    A computationally feasible parametric procedure for unsupervised learning has been given by Agrawala [1]. The procedure eliminates the computational difficulties associated with updating using a mixture density by making use of a probabilistic labeling scheme. Shanmugam [2] has given a similar parametric procedure using probabilistic labeling for the more general problem of imperfectly supervised ... View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Probability of error in digital communication systems with intersymbol interference and dependent symbols (Corresp.)

    Publication Year: 1974, Page(s):663 - 668
    Cited by:  Papers (2)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (724 KB)

    An upper and lower bound to the probability of error is presented for a digital communication system with dependent symbols affected by additive noise and intersymbol interference. Explicitly considered are two systems in which independent binary symbols are encoded into ternary dependent symbols, i.e., a bipolar code and a dicode. The bounds practically coincide under a proper choice of certain i... View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • The separability of Gray code (Corresp.)

    Publication Year: 1974, Page(s): 668
    Cited by:  Papers (13)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (140 KB)

    It is proved that, when binary strings are Gray coded into multilevel signals, to cause an error of<tex>m</tex>bits, the multilevel signal must have an error of more than<tex>2^m /3</tex>. This generalizes the well-known result that a one-level error in the signal generates only a one-bit error in the binary string. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Rate-distortion functions for nonhomogeneous Poisson processes (Corresp.)

    Publication Year: 1974, Page(s):669 - 672
    Cited by:  Papers (2)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (651 KB)

    Information rates for nonhomogeneous Poisson counting processes are derived. A source coding theorem is proved and rate-distortion functions are obtained for the reconstruction of the sample functions of the latter sources. Distortion measures which depend upon the magnitude of the error as well as on a temporal weighting function proportional to the instantaneous intensity of the source are assum... View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Rate-distortion theory for ergodic sources with side information (Corresp.)

    Publication Year: 1974, Page(s):672 - 675
    Cited by:  Papers (4)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (657 KB)

    The definition of the rate-distortion function is extended to the case of a stationary-ergodic source with side information, and the appropriate coding theorem is proved. Inequalities between the joint, marginal, and conditional rate-distortion functions for ergodic processes are given, and their implications in terms of universal coding are discussed. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • An upper bound on the entropy of run-length coding (Corresp.)

    Publication Year: 1974, Page(s):675 - 676
    Cited by:  Papers (4)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (200 KB)

    An upper bound on the entropy per run in binary run-length coding is<tex>a \loga - (a - 1)\log (a - 1)</tex>, where<tex>a</tex>is the average run length. This upper bound is attained by a time-quantized Poisson square wave. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A low-rate improvement on the Elias bound (Corresp.)

    Publication Year: 1974, Page(s):676 - 678
    Cited by:  Papers (12)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (343 KB)

    An upper bound on the minimum distance of binary blocks codes, which is superior to Elias' bound for<tex>R &lt; 0.0509^+</tex>, is obtained. The new hound has the same derivative<tex>(-\infty)</tex>at<tex>R = 0</tex>as Gilbert's lower bound. (Elias' bound has derivative<tex>-\ln 2</tex>at<tex>R = 0)</tex>. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A Gilbert-Varshamov bound for quasi-cycle codes of rate 1/2 (Corresp.)

    Publication Year: 1974, Page(s): 679
    Cited by:  Papers (62)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (125 KB)

    It is shown that there exist arbitrarily long quasi-cyclic<tex>(2k,k)</tex>binary codes that meet a bound slightly weaker than the Gilbert-Varshamov bound. This is a refinement of the result of Chen, Peterson, and Weldon [1]. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Construction and decomposition of cyclic codes of composite length (Corresp.)

    Publication Year: 1974, Page(s):680 - 683
    Cited by:  Papers (8)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (627 KB)

    A construction of cyclic codes in which an "inner" cyclic code is concatenated with appropriately chosen "outer" cyclic codes is presented. An example of the construction that improves on BCH codes of low rate is provided. Conversely, any cyclic code of length<tex>n_1 n_2</tex>can be constructed by this construction, where<tex>n_1</tex>and<tex>n_2</tex>are relat... View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.

Aims & Scope

IEEE Transactions on Information Theory publishes papers concerned with the transmission, processing, and utilization of information.

Full Aims & Scope

Meet Our Editors

Editor-in-Chief
Alexander Barg

Department of Electrical and Computer Engineering and the Institute for Systems Research, University of Maryland

email: abarg-ittrans@ece.umd.edu