IEEE Transactions on Information Theory

An explanation is provided for the prevalence of apparently convergent relative frequencies in random sequences. The explanation is based upon the computational-complexity characterization of a random sequence. Apparent convergence is shown to be attributable to a surprising consequence of the selectivity with which relative frequency arguments are applied; it is a consequence of data handling rat... View full abstract»

A stochastic approximation algorithm is developed for estimating a mixture of normal density functions with unknown means and unknown variances. The algorithm minimizes an information criterion that has interesting properties for density approximations. The conditions on the convergence of this nonlinear estimation algorithm are discussed, and a numerical example is presented. View full abstract»

A study of the use of entropy as a criterion function for analyzing the performance of sampled-data estimating systems is presented, and performance bounds are obtained for a broad class of such systems. The general result is that the difference between the entropy of the signal to be estimated and the entropy of its estimate based on the output of a noisy sensor can never be larger than the mutua... View full abstract»

A decision-directed receiver (DDR) uses previous outputs (decisions) to estimate unknown parameters and, on the basis of these estimates, modifies the detector structure for subsequent decisions. Although the DDR is less complex to instrument than other adaptive schemes, inherent in the decision-directed approach is the possibility of a runaway. This occurs when the detector commits a sequence of ... View full abstract»

We give a comprehensive discussion of the structure of the likelihood ratio (LR) for discrimination between two Gaussian processes, one of which is white. Several more general problems can be reduced, usually by differentiation, to this form. We shall show that nonsingular detection problems of this form can always be interpreted as problems of the apparently more special "signal-in-noise" type, w... View full abstract»

The relationship between distance measures and asymptotic relative efficiency is discussed. It is shown that the ratio of the Bhattacharyya distance or<tex>J</tex>divergences of two test statistics is equivalent to asymptotic relative efficiency. Two-input systems are discussed as examples, and the performances of the polarity coincidence correlator (PCC) and the correlator are discuss... View full abstract»

This paper is concerned with the mathematical aspect of a detection problem (a random signal in white Gaussian noise). Specifically, we obtain a sufficient condition for nonsingular detection and derive a likelihood-ratio expression in terms of least-mean-square estimates. The problem itself is old, and the likelihood-ratio expression is also well known. The contribution of this paper is a relativ... View full abstract»

It is shown that the asymptotic behavior of the tail of the sum distribution of a finite number of individually log-normal distributed variates displays the log-normal character of those variates with maximum logarithmic variance. A quantitative definition of the sum-distribution "tail" is established in terms of upper bounds on the relative error or deviation from the asymptotic log-normal proper... View full abstract»

Let<tex>\{q^(1) (t)\}</tex>, the signal, be a complex Gaussian process corrupted by additive Gaussian noise<tex>\{q^(2) (t) \}</tex>. Observations on<tex>p(t)q(t)</tex>and<tex>p(t) q^(2) (t)</tex>are assumed to be available where<tex>p(t)</tex>is a smooth weighting function and<tex>q = q^(1) + q^(2)</tex>. Using the Fourier tr... View full abstract»

The detection performance of some one-sample nonparametric rank tests for signals in Gaussian noise exhibiting a characteristic trend is presented in this paper for a finite number of observations and asymptotically. The tests considered are the Kendall tau, the Spearman rho, and a version of the<tex>c_1</tex>normal-scores test; both coherent and incoherent signal processing are consid... View full abstract»

We are concerned with the detection of a set of<tex>M</tex>messages that are transmitted over a channel disturbed by chaotic thermal noise when quantum effects in the communication systems are taken into account. Our attention is restricted to the special case in which the density operators specifying the state of the received field are commutative. In particular, the performance of tw... View full abstract»

The first portion of this paper is tutorial. Beginning with a standard definition of an abstract pattern-recognition machine, "learning" is given a mathematical meaning and the distinction is made between supervised and unsupervised learning. The bibliography will help the interested reader retrace the history of learning in pattern recognition. The exposition now focuses attention on unsupervised... View full abstract»

Four realizations of interleavers that reorder a sequence of symbols so that no contiguous sequence of<tex>n_2</tex>symbols in the reordered sequence contains any pair of symbols that were separated by fewer than<tex>n_1</tex>, symbols in the original ordering are introduced. For any<tex>n_1</tex>and<tex>n_2</tex>that satisfy an appropriate relative ... View full abstract»

A refinement of Capon's asymptotic approximation of the eigenfunctions of covariance kernels with rational spectral densities is given. It is shown that this refinement is necessary if the approximation is to be useful in calculating the Karhunen-Loeve coefficients of a large class of functions. View full abstract»

In view of a certain randomized decision scheme, we simplify R&#233;nyi's result that concerns the conditions that a changing situation for binary decision must satisfy in order that the probability of decision errors tend to zero. We give its possible interpretation for a communication channel that has a tendency to become useless with time. View full abstract»

The well-known technique of detecting binary pulses in white noise and unknown dc drift by means of a high-pass filter followed by a bit-by-bit detector is not theoretically optimum. The optimum procedure is shown to require a rank-ordering operation on the received data. View full abstract»

Receiver gains are determined that minimize the average cost of excluding a Rayleigh-distributed signal from a receiver's finite dynamic range when a maximum-likelihood estimate of the unknown Rayleigh parameter is used. Optimum performance is independent of the unknown parameter and converges to "exact knowledge" as sample size increases. View full abstract»

An upper bound on the minimum probability of error for an equal-strength diversity channel is simply derived that improves a previously known bound by the factor<tex>[4(1 - p)]^(-1)</tex>,<tex>0 \leq p \leq \frac{1}{2}</tex>. View full abstract»

An interleaved fading channel whose state is known to the receiver is analyzed. The reliability function<tex>E(R)</tex>is obtained for rates<tex>R</tex>in the range<tex>R_c \leq R \leq C</tex>. The capacity is shown to be<tex>C = E_A \{ \frac{1}{2} \ln (1 + A^2 n)\}</tex>where<tex>A</tex>is a factor describing the fading mechanism and<... View full abstract»

A new general approach to the formulation of a non-parametric detector using dependent samples is introduced and applied to a space-diversity system employing dc signaling. A comparison based on a form of asymptotic relative efficiency is made between the new detector and a Mann-Whitney detector. Under certain conditions the new procedure demonstrates an improvement in transmission efficiency. View full abstract»

Let<tex>g(x)h(x) = x^n - 1, n = q^m - 1</tex>, and assume that<tex>h(x)</tex>contains a primitive factor<tex>f(x)</tex>of degree<tex>m</tex>. If<tex>V</tex>is the<tex>q</tex>-ary<tex>(n, k)</tex>cyclic code generated by<tex>g(x), U</tex>its subcode generated by<tex>g(x)f(x)</tex>, then it wi... View full abstract»