By Topic

Information Theory, IEEE Transactions on

Issue 2 • Date March 1970

Filter Results

Displaying Results 1 - 25 of 26
  • Comments on the Estimation of Distribution Functions [Corresp.]

    Publication Year: 1970 , Page(s): 226
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (139 KB)  

    First Page of the Article
    View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Review of 'Error Correcting Codes' (Mann, H. B., Ed.; 1968)

    Publication Year: 1970 , Page(s): 242 - 243
    Save to Project icon | Request Permissions | PDF file iconPDF (307 KB)  
    Freely Available from IEEE
  • Further results on the synchronization of binary cyclic codes (Corresp.)

    Publication Year: 1970 , Page(s): 238 - 241
    Cited by:  Papers (3)  |  Patents (2)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (560 KB)  

    It is shown that certain coset codes derived from binary cyclic codes can determine the magnitude of a synchronization error, as well as its direction by examining only the syndrome of the received n tuple. For such coset codes, therefore, the need for a search procedure to recover synchronism is eliminated. In addition, the range of slip that can be detected and corrected for noisy channels is extended. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Alias-free randomly timed sampling of stochastic processes

    Publication Year: 1970 , Page(s): 147 - 152
    Cited by:  Papers (30)  |  Patents (1)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1072 KB)  

    The notion of alias-free sampling is generalized to apply to random processes x(t) sampled at random times t_n ; sampling is said to be alias free relative to a family of spectra if any spectrum of the family can be recovered by a linear operation on the correlation sequence {r(n)} , where r(n) = E[x(l_{m+n}) \overline {x(t_m)}] . The actual sampling times t_n need not be known to effect recovery of the spectrum of x(t) . Various alternative criteria for verifying alias-free sampling are developed. It is then shown that any spectrum whatsoever can be recovered if {t_n} is a Poisson point process on the positive (or negative) half-axis. A second example of alias-free sampling is provided for spectra on a finite interval by periodic sampling (for t \leq t_o or t \geq t_o ) in which samples are randomly independently skipped (expunged), such that the average sampling rate is an arbitrarily small fraction of the Nyquist rate. A third example shows that randomly jittered sampling at the Nyquist rate is alias free. Certain related open questions are discussed. These concern the practical problems involved in estimating a spectrum from imperfectly known { r(n) } . View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Bounds on performance of optimum quantizers

    Publication Year: 1970 , Page(s): 172 - 184
    Cited by:  Papers (11)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1632 KB)  

    A quantizer Q divides the range [0, 1] of a random variable x into K quantizing intervals the i th such interval having length \Delta x_i . We define the quantization error for a particular value of x (unusually) as the length of the quantizing interval in which x finds itself, and measure quantizer performance (unusually) by the r th mean value of the quantizing interval lengths M_r (Q) = \overline {\Delta x^{r^{1/r}}} , averaging with respect to the distribution function F of the random variable x . Q_1 is defined to be an optimum quantizer if M_r (Q_1) \leq M_r (Q) for all Q . The unusual definitions restrict the results to bounded random variables, but lead to general and precise results. We define a class Q^{\ast } of quasi-optimum quantizers; Q_2 is in Q^{\ast } if the different intervals \Delta x_i make equal contributions to the mean r th power of the interval size so that \Pr { \Delta x_i } \Delta x_{i^{r}} is constant for all i . Theorems 1, 2, 3, and 4 prove that Q_2 \in Q^{\ast } exists and is unique for given F, K , and r : that 1 \geq KM_r (Q_2) \geq KM_r (Q_1) \geq I_r , where I_r = {\int_0^{1} f (x)^p dx}^ {1/q}, f is the density of the absolutely continuous part of the distribution function F of x, p = 1/(1+ r) , and q = r /(1 + r) : that \lim KM_r (Q_2) = I_r as K \rightarrow \infty ; and that if KM_r (Q) = I_r for finite K , then Q=Q^{\ast } . View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • On the representation of continuous parameter processes by a sequence of random variables

    Publication Year: 1970 , Page(s): 139 - 141
    Cited by:  Papers (4)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (448 KB)  

    This paper examines the question of representing a continuous parameter random process { x_t, t \in T } by a sequence of random variables "without loss of information." The principal result is that such a representation by expansion coefficients relative to a basis { \phi_i } of mathcal{L}_2(T) is always possible, regardless of the orthogonality of { \phi_i } and of the boundedness of the time interval T , provided only that the process is continuous in probability and almost every sample path has finite energy. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Some difference triangles for constructing self-orthogonal codes (Corresp.)

    Publication Year: 1970 , Page(s): 237 - 238
    Cited by:  Papers (5)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (272 KB)  

    First Page of the Article
    View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Geometry of optimum incoherent detection

    Publication Year: 1970 , Page(s): 202 - 205
    Cited by:  Papers (1)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (712 KB)  

    The problem of distinguishing between two narrow-band signals in the presence of additive Gaussian noise when the forms of both signals are known except for phase is basically four dimensional, as the lack of phase information turns signal vectors into signal circles. The geometry of signal vectors and of the planes of signal circles is shown to account for the optimality of matched-filter detection. In the case of equal-energy equiprobable signals, it leads to the conclusion that a large variety of different pairs Of filters will perform exactly like a pair of matched filters. One such pair has orthogonal impulse responses, and its performance is, therefore, relatively easy to determine. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Flicker effect and television compression

    Publication Year: 1970 , Page(s): 214 - 218
    Cited by:  Papers (3)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (984 KB)  

    The perception of flicker by the human eye depends upon the picture content, and this paper describes a relation between the critical flicker frequency f_c and the spatial frequency f_x . Provided that the visual picture is presented in such a way that each spatial component, presented at a frequency f_p , obeys the inequality f_p \geq f_c = f(f_x) = f_0 \exp (-k f_x) , no flicker effect is perceived. The constants f_0 and k are weakly dependent upon the brightness and picture area; they are a little more strongly dependent upon the contrast ratio and the type of picture. The effect can be exploited in motion pictures or television by presenting high-quality pictures interleaved with low-quality ones. In the latter, some high spatial frequencies would be omitted. For television, band reduction of a face by 9:1 is possible. Practical consideration may reduce this compression ratio to a little short of 2:1. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • On the representation of a continuous stochastic intensity by Poisson shot noise

    Publication Year: 1970 , Page(s): 142 - 147
    Cited by:  Papers (3)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (968 KB)  

    In many applications a Poisson shot noise (PSN) process is said to statistically "represent" its intensity process. In this paper an investigation is made of the relationship between a PSN process and its intensity, when the latter is a sample function of a continuous stochastic process. The difference of the moments and the mean-square difference between the two processes are examined. The continuity assumption on the intensity permits the development of a sequence of moment relationships in which the effect of the PSN parameters can be seen. The results simplify and afford some degree of physical interpretation when the component functions of the PSN are "rectangular," or when the intensity process does not vary appreciably over their time width. An integral equation is derived that defines the component function that minimizes the mean-square difference between the two processes. It is shown that a "degenerate" form of component function induces complete statistical equality of the two processes. The problem has application to optical communication systems using photodetectors. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Nonbinary random error-correcting codes (Corresp.)

    Publication Year: 1970 , Page(s): 236 - 237
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (240 KB)  

    Primitive BCH codes with symbols from GF(q) and designed distance d have parameter values begin{align} text{block length} &= n = q^m - 1 \ text{check symbols/block} &= r leq m(d - 1) end{align} where m is any positive integer. For many nonbinary BCH codes (called maximally redundant codes), the maximum number of check symbols per block is required, i.e. r = m(d - 1) . Conditions whereby a primitive nonbinary BCH code is maximally redundant are discussed. It is shown that a class of codes exists, with symbols from GF(q) , based upon doubly lengthened Reed-Solomon codes over GF(q^m) , having parameter values begin{align} text{block length} &= n = m(q^m + 1) \ text{check symbols/block} &= r = m(d - 1) \ text{designed distance} &= d end{align} where again m is any positive integer. Thus this class of codes extends the block length of maximally redundant codes by a multiplicative factor exceeding m , while retaining the same designed distance and same number of check symbols. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • The inversion of covariance matrices by finite Fourier transforms (Corresp.)

    Publication Year: 1970 , Page(s): 230 - 232
    Cited by:  Papers (8)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (400 KB)  

    A matrix series is derived that converges to the inverse of a covariance matrix. The members of the series are derived from a circular matrix that can be inverted by taking finite Fourier transforms. An example of the method is presented. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Nonlinear estimation with quantized measurements--PCM, predictive quantization, and data compression

    Publication Year: 1970 , Page(s): 152 - 161
    Cited by:  Papers (18)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1344 KB)  

    Statistics conditioned on quantized measurements are considered in the general case. These results are specialized to Gaussian parameters and then extended to discrete-time linear systems. The conditional mean of the system's state vector may be found by passing the conditional mean of the measurement history through the Kalman filter that would be used had the measurements been linear. Repetitive use of Bayes' rule is not required. Because the implementation of this result requires lengthy numerical quadrature, two approximations are considered: the first is a power-series expansion of the probablity-density function; the second is a discrete-time version of a previously proposed algorithm that assumes the conditional distribution is normal. Both algorithms may be used with any memory length on stationary or nonstationary data. The two algorithms are applied to the noiseless-channel versions of the PCM, predictive quantization, and predictive-comparison data compression systems; ensemble-average performance estimates of the nonlinear filters are derived. Simulation results show that the performance estimates are quite accurate for most of the cases tested. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A note on filtering for independent increment processes (Corresp.)

    Publication Year: 1970 , Page(s): 224 - 226
    Cited by:  Papers (2)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (480 KB)  

    Using the equations of Fisher and Stear, a suboptimal filter for estimating the state of a linear system subject to non-Gaussian noise is developed. Some qualitative properties of this filter are derived. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Information rates of Wiener processes

    Publication Year: 1970 , Page(s): 134 - 139
    Cited by:  Papers (26)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (800 KB)  

    Rate distortion functions are calculated for time discrete and time continuous Wiener processes with respect to the mean squared error criterion. In the time discrete case, we find the interesting result that, for 0 \leq D \leq \sigma ^2 /4 , R(D) for the Wiener process is identical to R(D) for the sequence of zero mean independent normally distributed increments of variance sigma^2 whose partial sums form the Wiener process. In the time continuous case, we derive the explicit formula R(D) = 2 \sigma ^2 / ( \pi^2 D) , where \sigma ^2 is the variance of the increment daring a one-second interval. The resuiting R(D) curves are compared with the performance of an optimum integrating delta modulation system. Finally, by incorporating a delta modulation scheme in the random coding argument, we prove a source coding theorem that guarantees our R(D) curves are physically significant for information transmission purposes even though Wiener processes are nonstationary. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Signals that can be calculated from their ambiguity function

    Publication Year: 1970 , Page(s): 195 - 202
    Cited by:  Papers (4)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1072 KB)  

    A new lemma relates the analytic extensions of two time functions u(t) and v(t) to the Laplace transform of their ambiguity function \psi_{uv} . This lemma is used to derive necessary conditions for u and v from two bounds on the behavior of \psi_{uv} , at infinity. In particular, if the first bound is fulfilled, then u(z) and v(z) must be integral analytic functions. If both bounds are fulfilled, then u and v are each equal to \exp {- \pi t^2 } times a polynomial in t , and the two polynomials can be found from \psi_{uv} by comparing coefficients. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A few useful details about a known technique for factoring 1 +X^{2q-1} (Corresp.)

    Publication Year: 1970 , Page(s): 234 - 235
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (312 KB)  

    First Page of the Article
    View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Probing linear filters-- Signal design for the detection problem

    Publication Year: 1970 , Page(s): 167 - 171
    Cited by:  Papers (6)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (720 KB)  

    The problem of designing an optimum probing signal for discriminating between two known linear filters in the presence of noise is considered. It is shown that in the general case of colored noise, the optimum signal is the solution of two simultaneous integral equations. An expression for the probability of error is given, and some illustrative examples are presented. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Calculation of Fourier transforms on finite Abelian groups (Corresp.)

    Publication Year: 1970 , Page(s): 233 - 234
    Cited by:  Papers (8)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (344 KB)  

    A recent paper by Crimmins et al. deals with minimization of mean-square error for group codes by the use of Fourier transforms on groups. In this correspondence a method for representing the groups in a form suitable for machine calculation is shown. An efficient method for calculating the Fourier transform of a group is also proposed and its relationship to the fast Fourier transform is shown. For groups of characteristic two, the calculation requires only N \log _2 N additive operations where N is the order of the group. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Error probability bounds for systematic convolutional codes

    Publication Year: 1970 , Page(s): 219 - 224
    Cited by:  Papers (9)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (936 KB)  

    Upper and lower bounds on error probability for systematic convolutional codes are obtained. These bounds show that the probability of error for a systematic convolutional code is larger than the probability of error for a nonsystematic convolutional code of the same encoder-shift-register length. Two different interpretations of these results are given. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Fitting continuous probability density functions over [0, \infty ) using information theory ideas (Corresp.)

    Publication Year: 1970 , Page(s): 226 - 230
    Cited by:  Papers (3)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (728 KB)  

    The problem of estimating a probability density function p(x) over [0, \infty ) when several low-order moments are known is considered. A computational procedure, based on information theory, is used to obtain the maximum entropy estimates of p(x) in a number of cases. The situation when the first two moments only are known is considered in some detail. A table is included for estimating p(x) when given \mu_1 ^{\prime } , \mu_2 ^{\prime } with \mu_2 ^{\prime } \leq 2(\mu_1 ^{\prime }) ^ 2 . View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • An integral relevant to the detection of a signal in polarized Gaussian noise (Corresp.)

    Publication Year: 1970 , Page(s): 232
    Cited by:  Papers (1)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (136 KB)  

    An integral is calculated that arises while determining the probability of detecting a signal in polarized Gaussian noise and the probability of hitting a particular elliptical target with an imperfect weapon. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Signal design for efficient detection in dispersive channels

    Publication Year: 1970 , Page(s): 206 - 213
    Cited by:  Papers (7)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1200 KB)  

    Signal design for maximizing the efficiency of the Neyman-Pearson detection procedure in randomly dispersive media is investigated. The medium is modeled as a randomly time-varying linear filter; by viewing the filter transfer function as a homogeneous random field on the time-frequency plane, a second-order theory results that relates various second-order measures of the time and frequency structures of input and output processes. A signal design strategy is developed that dictates transmitting signals that produce output processes with degrees of freedom possessing a signal-to-noise ratio (SNR) in the vicinity of 2. A distribution of signal energy in the output time-frequency plane that achieves the proper SNR for each degree of freedom is deduced and is used to infer constraints on input ambiguity functions that maximize detection efficiency. The general structure of efficient input signals for both high and low SNR is briefly discussed. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Identification of a transition matrix of a Markov chain from noisy measurements of state

    Publication Year: 1970 , Page(s): 161 - 166
    Cited by:  Papers (4)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (720 KB)  

    We consider the problem of recursively estimating the transition matrix of a regular Markov chain with a finite number of states using only noisy measurements of the state. The measurement noise sequence is assumed to be independent with known mean and unknown variance. We also discuss the convergence rates and computational aspects of the algorithms and the methods of accelerating them. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • The two-armed-bandit problem with time-invariant finite memory

    Publication Year: 1970 , Page(s): 185 - 195
    Cited by:  Papers (24)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1680 KB)  

    This paper solves the classical two-armed-bandit problem under the finite-memory constraint described below. Given are probability densities p_0 and p_1 , and two experiments A and B . It is not known which density is associated with which experiment. Thus the experimental outcome Y of experiment A is as likely to be distributed according to p_0 as it is to be distributed according to p_1 . It is desired to sequentially choose an experiment to be performed on the basis of past observations according to the algorithm T_n = f(T_{n-1}, e_n, Y_n), e_n = e(T_{n-1}) , where T_n \in {1, 2, \cdots , m} is the state of memory at time n, e_n \in {A, B} is the choice of experiment, and Y_n , is the random variable observation. The goal is to maximize the asymptotic proportion r of uses of the experiment associated with density p_0 . Let l(y) = p_0 (y) / p_1 (y) , and let \bar{l} and \bar{\bar{l}} denote the almost everywhere greatest lower bound and least upper bound on l(y) . Let 1 = \max {\bar{\bar{l}}, 1/\bar{l}} . Then the optimal value of r , over all m -state algorithms (f, e) , will be shown to be l^{m-1} / (l^{m-1} + 1) . An e -optimal family of m -state algorithms will be demonstrated. In general, optimal algorithms do not exist, and e -optimal algorithms require artificial randomization. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.

Aims & Scope

IEEE Transactions on Information Theory publishes papers concerned with the transmission, processing, and utilization of information.

Full Aims & Scope

Meet Our Editors

Editor-in-Chief
Frank R. Kschischang

Department of Electrical and Computer Engineering