IEEE Transactions on Information Theory

Volume 16 Issue 4 • July 1970

Filter Results

Displaying Results 1 - 25 of 39
  • Probability of error, equivocation, and the Chernoff bound

    Publication Year: 1970, Page(s):368 - 372
    Cited by:  Papers (159)  |  Patents (8)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (513 KB)

    Relationships between the probability of error, the equivocation, and the Chernoff bound are examined for the two-hypothesis decision problem. The effect of rejections on these bounds is derived. Finally, the results are extended to the case of any finite number of hypotheses. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Learning with a probabilistic teacher

    Publication Year: 1970, Page(s):373 - 379
    Cited by:  Papers (38)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (896 KB)

    The Bayesian learning scheme is computationally infeasible for most of the unsupervised learning problems. This paper suggests a learning scheme, "learning with a probabilistic teacher," which works with unclassified samples and is computationally feasible for many practical problems. In this scheme a sample is probabilistically assigned with a class with appropriate probabilities computed using a... View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A theorem on conditional expectation

    Publication Year: 1970, Page(s):379 - 381
    Cited by:  Papers (9)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (269 KB)

    A statistic often encountered in various estimation problems is the conditional ensemble average of the time derivative of a random signal given the signal. It turns out that for a very large class of random signals this statistic is equal to zero. This is a rather surprising result and as far as can be determined has not been precisely stated and rigorously proven. A precise statement and a rigor... View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Steady-state error in adaptive mean-square minimization

    Publication Year: 1970, Page(s):382 - 385
    Cited by:  Papers (17)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (536 KB)

    This paper considers the steady-state mean-square error when an adaptive linear estimator is used on a stationary time series. The estimator weights are adjusted periodically by moving a small increment in the direction of the estimated gradient. Under very general conditions the asymptotic mean-square error is bounded and under more restrictive conditions is evaluated exactly. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Bootstrapping the generalized least-squares estimator in colored Gaussian noise with unknown covariance parameters

    Publication Year: 1970, Page(s):385 - 392
    Cited by:  Papers (1)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (1120 KB)

    It is demonstrated that in problems involving the estimation of linear regression parameters in colored Gaussian noise, the simple least-squares estimator can be significantly suboptimal. When the noise covariance function can be described as a known function of a finite number of unknown nonrandom parameters, it is possible to take advantage of this information to improve upon the least-squares e... View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A further note on a general likelihood formula for random signals in Gaussian noise

    Publication Year: 1970, Page(s):393 - 396
    Cited by:  Papers (25)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (522 KB)

    A general likelihood ratio formula of the author's for the detection of signals in independent white Gaussian noise is extended to allow a "one-sided" dependence in which only the future white noise is required to be independent of past signal and noise. The assumption of Gaussian additive noise is also somewhat relaxed. The proof is based on some recent martingale theorems and on the concept of t... View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Application of state-variable techniques to optimal feature extraction--- Multichannel analog data

    Publication Year: 1970, Page(s):396 - 406
    Cited by:  Papers (12)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (1489 KB)

    In problems of pattern recognition and signal detection, one of the most important tasks is that of finding practical ways of preprocessing the data to eliminate complexity, redundancy, and irrelevancy. In this paper it is assumed that a vector wavefonn is received during an interval<tex>[t_o, t_f]</tex>. The waveform is considered to be a sample of a nonstationary vector random proces... View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Transmission of noisy information to a noisy receiver with minimum distortion

    Publication Year: 1970, Page(s):406 - 411
    Cited by:  Papers (105)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (814 KB)

    This paper is concerned with the transmission of information with a fidelity criterion where the source output may be distorted prior to encoding and, furthermore, where the output of the decoder may be distorted prior to its delivery to the final destination. The criterion for optimality is that the normalized average of the squared norm of the difference between the<tex>T</tex>- seco... View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Information rates of autoregressive processes

    Publication Year: 1970, Page(s):412 - 421
    Cited by:  Papers (87)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (1374 KB)

    The rate distortion function<tex>R(D)</tex>is calculated for two time-discrete autoregressive sources--the time-discrete Gaussian autoregressive source with a mean-square-error fidelity criterion and the binary-symmetric first-order Markov source with an average probability-of-error per bit fidelity criterion. In both cases it is shown that<tex>R(D)</tex>is bounded below by... View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Intersymbol interference on a continuous-time Gaussian channel

    Publication Year: 1970, Page(s):422 - 429
    Cited by:  Papers (3)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (1120 KB)

    The effect on the probability of error from intersymbol interference caused by a repeated use of a continuous-time Gaussian channel is examined. The model assumes that signals that are power constrained are put through a linear filter after which Gaussian noise is added to produce the received signal. One can view the linear filter as a frequency constraint on the inputs. Because of the memory in ... View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Mean-square-error optimum coset leaders for group codes

    Publication Year: 1970, Page(s):429 - 432
    Cited by:  Papers (14)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (556 KB)

    Crimmins et al. [1] showed how to minimize mean-square error over all 1 - 1 mappings of digital data into binary group code elements when the a priori probability of the data is uniform. This paper shows how to optimize simultaneously over all possible choices of coset leaders for the detection scheme. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Block codes for statistical synchronization

    Publication Year: 1970, Page(s):432 - 438
    Cited by:  Papers (5)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (846 KB)

    The performance of a class of block codes with unbounded synchronization delay, though finite average synchronization delay, is analyzed. Basically the synchronizer inspects the code symbol stream for the first occurrence of one of a set of sequences that appear in only one timing position relative to true sync. The receiver can be implemented easily with shift registers and associated logic. The ... View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Mechanization of codes with bounded synchronization delays

    Publication Year: 1970, Page(s):438 - 446
    Cited by:  Papers (7)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (1337 KB)

    Bounded synchronization delay codes have the property that no proper cyclic rearrangement of the letters of a codeword is another codeword. Because of this property research in code design has centered on criteria for selecting one word from each nonperiodic cyclic equivalence class to satisfy various additional constraints. This is all that is necessary when it is feasible for the encoder and dec... View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Further results on product codes

    Publication Year: 1970, Page(s):446 - 451
    Cited by:  Papers (17)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (974 KB)

    A new class of codes, called product generator codes, which are similar to Elias's iterated codes, are investigated. An important subclass of these codes is the generalized Reed-Muller codes. If the original codes that are iterated to produce a product code are<tex>L_1</tex>and<tex>L_2</tex>-step orthogonalizable, then the product code is (<tex>L_1 + L_2 - 1</tex&g... View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Further results on cyclic product codes

    Publication Year: 1970, Page(s):452 - 459
    Cited by:  Papers (22)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (836 KB)

    Cyclic product codes are useful for two reasons. First, they impart a great deal of algebraic structure to a subclass of the class of cyclic codes. Second, because they can be formulated in terms of much shorter (component) codes, their decoding may be considerably simpler than many other types of codes. In this paper both of the properties of cyclic product codes are developed. It is shown that t... View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Analysis of decoders for convolutional codes by stochastic sequential machine methods

    Publication Year: 1970, Page(s):460 - 469
    Cited by:  Papers (15)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (1406 KB)

    In this paper, the decoder of a convolutional code is modeled as an autonomous stochastic sequential machine and finite Markov chain theory applied to obtain a precise expression for<tex>P_{FD} (u)</tex>, the probability of error associated with the feedback decoding of the<tex>u</tex>th subblock of information digits. The analysis technique developed extends directly to an... View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Arithmetic norms and bounds of the arithmetic AN codes

    Publication Year: 1970, Page(s):470 - 476
    Cited by:  Papers (4)  |  Patents (3)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (701 KB)

    Properties of integers, related to the generation of the arithmetic AN codes, are investigated in this paper. A programmable algorithm for the computation of the binary norm of an arbitrary integer is developed. A table of norms of the natural numbers is generated and from this the distribution of integers of a given norm is found. These results are used to compute bounds on the size of an<tex&... View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Information channel capacity of a photographic film (Corresp.)

    Publication Year: 1970, Page(s):477 - 480
    Cited by:  Papers (1)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (660 KB)

    A different approach toward the information channel capacity of a photographic film is presented. It is based on the total number of possible combinations of the smallest distinguishable regions in a piece of photographic film. The information channel capacity of a hologram is also discussed. It is found that the information capacity of a hologram is somewhat lower than that of a photographic film... View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Reexamination of the prefiltering problem in a state-variable formulation (Corresp.)

    Publication Year: 1970, Page(s):480 - 484
    Cited by:  Papers (4)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (702 KB)

    Optimal (least-mean-square linear) filtering of random signals prior to sampling may be represented as a Wiener-Kalman state-variable filter followed by a coder that synthesizes the signal to be sampled as a linear combination of the estimated states. Although the prefilter doubles the number of states of the overall presampling signal process, the postsampling reconstruction filter need only mode... View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Factorization of spectra by discrete Fourier transforms (Corresp.)

    Publication Year: 1970, Page(s):484 - 485
    Cited by:  Papers (9)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (294 KB)

    A discrete Fourier transform method for factoring arbitrary spectral density functions is presented. The factorization can be implemented in a straightforward and efficient manner, and it does not require that the spectra be rational. An expression for the absolute error is also presented. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • On finite-memory recursive filters (Corresp.)

    Publication Year: 1970, Page(s):486 - 487
    Cited by:  Papers (5)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (213 KB)

    It has not been noted in the literature that finite-memory least-squares, recursive filters are unstable, and hence unsuited for a large number of recursions. This correspondence modifies the finite-memory least-squares recursive filters to allow for stable solutions. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Estimating the amplitude of a sinusoid from noisy random samples (Corresp.)

    Publication Year: 1970, Page(s):487 - 489
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (351 KB)

    The problem of estimating the amplitude of a sinusoid from noisy random samples is examined. The maximum-likelihood procedure, applied in the strong-signal case, yields a simple (albeit nonlinear) estimate. The mean-squared estimation error is obtained as a function of the sample size as well as the signal-to-noise ratio. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A note on the finite Walsh transform (Corresp.)

    Publication Year: 1970, Page(s):489 - 491
    Cited by:  Papers (21)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (398 KB)

    The theory of the finite Walsh transform is presented as a background to current interest in the fast Walsh transform algorithm. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Bandwidth requirements for Walsh functions (Corresp.)

    Publication Year: 1970, Page(s):491 - 493
    Cited by:  Papers (8)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (411 KB)

    The power-density spectrum for the Walsh function of binary order<tex>k</tex>is calculated. The model analyzed considers the Walsh function as a random sequence modulating a<tex>k</tex>th-order Rademacher function. The total number of Walsh functions of binary order<tex>k</tex>and less is found to be about<tex>\frac{1}{6}</tex>of the overbound 2.4<... View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Note on a partition for a decision feedback strategy (Corresp.)

    Publication Year: 1970, Page(s):493 - 494
    Cited by:  Papers (1)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (355 KB)

    A partition of the channel output space, which has the property that no other partition yields both a smaller probability of erasure and a smaller probability of error, is obtained and compared with a previous result due to Forney. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.

Aims & Scope

IEEE Transactions on Information Theory publishes papers concerned with the transmission, processing, and utilization of information.

Full Aims & Scope

Meet Our Editors

Editor-in-Chief
Alexander Barg

Department of Electrical and Computer Engineering and the Institute for Systems Research, University of Maryland

email: abarg-ittrans@ece.umd.edu