By Topic

Information Theory, IEEE Transactions on

Issue 3 • Date May 1970

Filter Results

Displaying Results 1 - 24 of 24
  • Review of 'Knowing and Guessing, A Quantitative Study of Inference and Information' (Watanabe, S.; 1969)

    Publication Year: 1970 , Page(s): 361 - 362
    Save to Project icon | Request Permissions | PDF file iconPDF (379 KB)  
    Freely Available from IEEE
  • Review of 'Signal Theory' (Franks, L. E.; 1969)

    Publication Year: 1970 , Page(s): 362
    Save to Project icon | Request Permissions | PDF file iconPDF (153 KB)  
    Freely Available from IEEE
  • A Gaussian channel with slow fading (Corresp.)

    Publication Year: 1970 , Page(s): 353 - 355
    Cited by:  Papers (53)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (432 KB)  

    An interleaved fading channel whose state is known to the receiver is analyzed. The reliability function E(R) is obtained for rates R in the range R_c \leq R \leq C . The capacity is shown to be C = E_A { frac{1}{2} \ln (1 + A^2 n)} where A is a factor describing the fading mechanism and u is the signal-to-noise ratio per dimension. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Statistically optimum AGC for Rayleigh-distributed signals of unknown strength (Corresp.)

    Publication Year: 1970 , Page(s): 351 - 352
    Cited by:  Papers (1)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (280 KB)  

    Receiver gains are determined that minimize the average cost of excluding a Rayleigh-distributed signal from a receiver's finite dynamic range when a maximum-likelihood estimate of the unknown Rayleigh parameter is used. Optimum performance is independent of the unknown parameter and converges to "exact knowledge" as sample size increases. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • An improved error bound for equal-strength diversity channels (Corresp.)

    Publication Year: 1970 , Page(s): 352 - 353
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (272 KB)  

    An upper bound on the minimum probability of error for an equal-strength diversity channel is simply derived that improves a previously known bound by the factor [4(1 - p)]^(-1) , 0 \leq p \leq frac{1}{2} . View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Analysis of a decision-directed receiver with unknown priors

    Publication Year: 1970 , Page(s): 270 - 276
    Cited by:  Papers (25)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1032 KB)  

    A decision-directed receiver (DDR) uses previous outputs (decisions) to estimate unknown parameters and, on the basis of these estimates, modifies the detector structure for subsequent decisions. Although the DDR is less complex to instrument than other adaptive schemes, inherent in the decision-directed approach is the possibility of a runaway. This occurs when the detector commits a sequence of decision errors resulting in a degradation of parameter estimates, which, in turn, results in a further deterioration of detector performance. Because of the dependencies introduced by the learning process, runaway is difficult to analyze. In this paper, a DDR with unknown a priori probabilities is considered. The priors are estimated by the relative frequency of decisions of that event. For binary detection, it is shown that there is a positive probability of a runaway (the estimates converge to 1 or 0), which equals 1 if the signal-to-noise ratio is below a critical value. A tight bound on the probability of a runaway is obtained by approximating the learning process by a random walk with independent increments. The analysis demonstrates that a runaway is quite improbable even for moderate signal-to-noise ratios. The analysis is extended to multiple signals and to the situation where the estimates of the priors are updated continuously through exponential weighting rather than allowed to converge. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Nonparametric detection using dependent samples (Corresp.)

    Publication Year: 1970 , Page(s): 355 - 358
    Cited by:  Papers (8)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (680 KB)  

    A new general approach to the formulation of a non-parametric detector using dependent samples is introduced and applied to a space-diversity system employing dc signaling. A comparison based on a form of asymptotic relative efficiency is made between the new detector and a Mann-Whitney detector. Under certain conditions the new procedure demonstrates an improvement in transmission efficiency. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Asymptotic eigenfunctions of covariance kernels with rational spectral densities (Corresp.)

    Publication Year: 1970 , Page(s): 346 - 347
    Cited by:  Papers (2)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (264 KB)  

    A refinement of Capon's asymptotic approximation of the eigenfunctions of covariance kernels with rational spectral densities is given. It is shown that this refinement is necessary if the approximation is to be useful in calculating the Karhunen-Loeve coefficients of a large class of functions. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • On the apparent convergence of relative frequency and its implications

    Publication Year: 1970 , Page(s): 251 - 257
    Cited by:  Papers (2)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (896 KB)  

    An explanation is provided for the prevalence of apparently convergent relative frequencies in random sequences. The explanation is based upon the computational-complexity characterization of a random sequence. Apparent convergence is shown to be attributable to a surprising consequence of the selectivity with which relative frequency arguments are applied; it is a consequence of data handling rather than an underlying law or good fortune. The consequences of this understanding for probability and its applications are indicated. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Nonsingular detection and likelihood ratio for random signals in white Gaussian noise

    Publication Year: 1970 , Page(s): 291 - 298
    Cited by:  Papers (4)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1008 KB)  

    This paper is concerned with the mathematical aspect of a detection problem (a random signal in white Gaussian noise). Specifically, we obtain a sufficient condition for nonsingular detection and derive a likelihood-ratio expression in terms of least-mean-square estimates. The problem itself is old, and the likelihood-ratio expression is also well known. The contribution of this paper is a relatively elementary and self-contained derivation of the likelihood-ratio expression as well as the nonsingularity condition. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Detection performance of some nonparametric rank tests and an application to radar

    Publication Year: 1970 , Page(s): 309 - 318
    Cited by:  Papers (12)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1248 KB)  

    The detection performance of some one-sample nonparametric rank tests for signals in Gaussian noise exhibiting a characteristic trend is presented in this paper for a finite number of observations and asymptotically. The tests considered are the Kendall tau, the Spearman rho, and a version of the c_1 normal-scores test; both coherent and incoherent signal processing are considered. The asymptotic performance is determined through a simple extension of existing analytical results, whereas for a finite number of observations a direct Monte Carlo simulation is used. The latter requires a preliminary determination of false-alarm probability versus decision threshold. For the Spearman rho and the c_1 tests an importance-sampling technique is used in a Monte Carlo simulation to determine the required results. Finally an application of the Kendall tau test to conventional radar is considered, in which the characteristic trend due to the amplitude modulation of the received train of echoes by the scanning of the antenna is used for detecting the presence of a target. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Realization of optimum interleavers

    Publication Year: 1970 , Page(s): 338 - 345
    Cited by:  Papers (87)  |  Patents (60)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1232 KB)  

    Four realizations of interleavers that reorder a sequence of symbols so that no contiguous sequence of n_2 symbols in the reordered sequence contains any pair of symbols that were separated by fewer than n_1 , symbols in the original ordering are introduced. For any n_1 and n_2 that satisfy an appropriate relative primeness condition, these interleavers are optimum in the sense that of all possible interleavers providing the indicated symbol separation, one of these four realizations achieves both the minimum possible encoding delay and the minimum possible combined storage capacity for the interleaver and its unscrambler. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Reliability of quantum-mechanical communication systems

    Publication Year: 1970 , Page(s): 319 - 329
    Cited by:  Papers (9)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1680 KB)  

    We are concerned with the detection of a set of M messages that are transmitted over a channel disturbed by chaotic thermal noise when quantum effects in the communication systems are taken into account. Our attention is restricted to the special case in which the density operators specifying the state of the received field are commutative. In particular, the performance of two special communication systems is evaluated. For a system in which orthogonal signals with known amplitudes and random phases are transmitted over an additive white Gaussian channel, the structure of an optimum receiver is found. Expressions for the system reliability function and channel capacity are derived. For a system in which orthogonal signals are transmitted over a Rayleigh fading channel, the optimum performance is obtained. The optimum degree of diversity for an equal-strength diversity system is found numerically as a function of the average thermal-noise energy and information rate. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Likelihood ratios for Gaussian processes

    Publication Year: 1970 , Page(s): 276 - 288
    Cited by:  Papers (26)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (2072 KB)  

    We give a comprehensive discussion of the structure of the likelihood ratio (LR) for discrimination between two Gaussian processes, one of which is white. Several more general problems can be reduced, usually by differentiation, to this form. We shall show that nonsingular detection problems of this form can always be interpreted as problems of the apparently more special "signal-in-noise" type, where the cross-covariance function of the signal and noise must be of a special "one-sided" form. Moreover, the LR for this equivalent problem can be written in the same form as that for known signals in white Gaussian noise, with the causal estimate of the signal process replacing the known signal. This single formula will be shown to be equivalent to a variety of other formulas, including all those previously known. The proofs are based on a resolvent identity and on a representation theorem for second-order processes, both of which have other applications. This paper also contains a discussion of the various stochastic integrals and infinite determinants that arise in Gaussian detection problems View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • On the weight distribution of cyclic codes (Corresp.)

    Publication Year: 1970 , Page(s): 358
    Cited by:  Papers (1)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (160 KB)  

    Let g(x)h(x) = x^n - 1, n = q^m - 1 , and assume that h(x) contains a primitive factor f(x) of degree m . If V is the q -ary (n, k) cyclic code generated by g(x), U its subcode generated by g(x)f(x) , then it will be shown that the weight distribution of V can be obtained from the weight distribution of U and its coset U + g(x) . View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • On zero error probability of binary decisions (Corresp.)

    Publication Year: 1970 , Page(s): 347 - 348
    Cited by:  Papers (1)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (280 KB)  

    In view of a certain randomized decision scheme, we simplify Rényi's result that concerns the conditions that a changing situation for binary decision must satisfy in order that the probability of decision errors tend to zero. We give its possible interpretation for a communication channel that has a tendency to become useless with time. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Stochastic estimation of a mixture of normal density functions using an information criterion

    Publication Year: 1970 , Page(s): 258 - 263
    Cited by:  Papers (18)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (856 KB)  

    A stochastic approximation algorithm is developed for estimating a mixture of normal density functions with unknown means and unknown variances. The algorithm minimizes an information criterion that has interesting properties for density approximations. The conditions on the convergence of this nonlinear estimation algorithm are discussed, and a numerical example is presented. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • On estimating spectral moments in the presence of colored noise

    Publication Year: 1970 , Page(s): 303 - 309
    Cited by:  Papers (11)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (936 KB)  

    Let {q^(1) (t)} , the signal, be a complex Gaussian process corrupted by additive Gaussian noise {q^(2) (t) } . Observations on p(t)q(t) and p(t) q^(2) (t) are assumed to be available where p(t) is a smooth weighting function and q = q^(1) + q^(2) . Using the Fourier transform of the samples of p(t)q(t) and p(t) q^(2) (t) , estimators are derived for estimating the mean frequency and spectral width of the unknown power spectrum of the unweighted signal process. The means and variances of these statistics are computed in general, and explicitly for nontrivial practical examples. Asymptotic formulas for the moment estimators as a function of the number of realizations, frequency resolution, signal-to-noise ratio and spectral width, and consistency of the estimators are some of the results that are discussed in detail. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Unsupervised learning and the identification of finite mixtures

    Publication Year: 1970 , Page(s): 330 - 338
    Cited by:  Papers (8)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1584 KB)  

    The first portion of this paper is tutorial. Beginning with a standard definition of an abstract pattern-recognition machine, "learning" is given a mathematical meaning and the distinction is made between supervised and unsupervised learning. The bibliography will help the interested reader retrace the history of learning in pattern recognition. The exposition now focuses attention on unsupervised learning. Carefully, it is explained how problems in this subject can be viewed as problems in the identification of finite mixtures, a statistical theory that has achieved some maturity. From this vantage point, it is demonstrated that identification theory implies unsupervised learning is possible in many important cases. The remaining sections present a general method for achieving unsupervised learning. Other authors have proposed schemes having greater computational convenience, but no method previously published is as inclusive as the one revealed here, which we demonstrate to be effective for all the many cases wherein unsupervised learning is known to be possible. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Rank-order statistics for optimum detection of binary signals in the presence of white noise and dc drift (Corresp.)

    Publication Year: 1970 , Page(s): 348 - 351
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (520 KB)  

    The well-known technique of detecting binary pulses in white noise and unknown dc drift by means of a high-pass filter followed by a bit-by-bit detector is not theoretically optimum. The optimum procedure is shown to require a rank-ordering operation on the received data. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Entropy analysis of estimating systems

    Publication Year: 1970 , Page(s): 264 - 270
    Cited by:  Papers (17)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1016 KB)  

    A study of the use of entropy as a criterion function for analyzing the performance of sampled-data estimating systems is presented, and performance bounds are obtained for a broad class of such systems. The general result is that the difference between the entropy of the signal to be estimated and the entropy of its estimate based on the output of a noisy sensor can never be larger than the mutual information between the sensor input and output. An interesting, but not totally satisfactory, sufficient condition for attainment of the bound is developed. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Tail of the distribution of sums of log-normal variates

    Publication Year: 1970 , Page(s): 299 - 302
    Cited by:  Papers (27)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (440 KB)  

    It is shown that the asymptotic behavior of the tail of the sum distribution of a finite number of individually log-normal distributed variates displays the log-normal character of those variates with maximum logarithmic variance. A quantitative definition of the sum-distribution "tail" is established in terms of upper bounds on the relative error or deviation from the asymptotic log-normal property. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Distance measures and asymptotic relative efficiency

    Publication Year: 1970 , Page(s): 288 - 291
    Cited by:  Papers (19)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (584 KB)  

    The relationship between distance measures and asymptotic relative efficiency is discussed. It is shown that the ratio of the Bhattacharyya distance or J divergences of two test statistics is equivalent to asymptotic relative efficiency. Two-input systems are discussed as examples, and the performances of the polarity coincidence correlator (PCC) and the correlator are discussed in terms of the distance measures of reduced data. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Computer results on the minimum distance of some binary cyclic codes (Corresp.)

    Publication Year: 1970 , Page(s): 359 - 360
    Cited by:  Papers (20)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (176 KB)  

    First Page of the Article
    View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.

Aims & Scope

IEEE Transactions on Information Theory publishes papers concerned with the transmission, processing, and utilization of information.

Full Aims & Scope

Meet Our Editors

Editor-in-Chief
Frank R. Kschischang

Department of Electrical and Computer Engineering