By Topic

Information Theory, IEEE Transactions on

Issue 2 • Date March 1970

Filter Results

Displaying Results 1 - 25 of 26
  • Full text access may be available. Click article title to sign in or learn about subscription options.
  • Review of 'Error Correcting Codes' (Mann, H. B., Ed.; 1968)

    Page(s): 242 - 243
    Save to Project icon | Request Permissions | PDF file iconPDF (307 KB)  
    Freely Available from IEEE
  • Full text access may be available. Click article title to sign in or learn about subscription options.
  • Identification of a transition matrix of a Markov chain from noisy measurements of state

    Page(s): 161 - 166
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (720 KB)  

    We consider the problem of recursively estimating the transition matrix of a regular Markov chain with a finite number of states using only noisy measurements of the state. The measurement noise sequence is assumed to be independent with known mean and unknown variance. We also discuss the convergence rates and computational aspects of the algorithms and the methods of accelerating them. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Signals that can be calculated from their ambiguity function

    Page(s): 195 - 202
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1072 KB)  

    A new lemma relates the analytic extensions of two time functionsu(t)andv(t)to the Laplace transform of their ambiguity functionpsi_{uv}. This lemma is used to derive necessary conditions foruandvfrom two bounds on the behavior ofpsi_{uv}, at infinity. In particular, if the first bound is fulfilled, thenu(z)andv(z)must be integral analytic functions. If both bounds are fulfilled, thenuandvare each equal toexp {- pi t^2 }times a polynomial int, and the two polynomials can be found frompsi_{uv}by comparing coefficients. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Geometry of optimum incoherent detection

    Page(s): 202 - 205
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (712 KB)  

    The problem of distinguishing between two narrow-band signals in the presence of additive Gaussian noise when the forms of both signals are known except for phase is basically four dimensional, as the lack of phase information turns signal vectors into signal circles. The geometry of signal vectors and of the planes of signal circles is shown to account for the optimality of matched-filter detection. In the case of equal-energy equiprobable signals, it leads to the conclusion that a large variety of different pairs Of filters will perform exactly like a pair of matched filters. One such pair has orthogonal impulse responses, and its performance is, therefore, relatively easy to determine. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Probing linear filters-- Signal design for the detection problem

    Page(s): 167 - 171
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (720 KB)  

    The problem of designing an optimum probing signal for discriminating between two known linear filters in the presence of noise is considered. It is shown that in the general case of colored noise, the optimum signal is the solution of two simultaneous integral equations. An expression for the probability of error is given, and some illustrative examples are presented. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Full text access may be available. Click article title to sign in or learn about subscription options.
  • Nonbinary random error-correcting codes (Corresp.)

    Page(s): 236 - 237
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (240 KB)  

    Primitive BCH codes with symbols fromGF(q)and designed distancedhave parameter values begin{align} text{block length} &= n = q^m - 1 \ text{check symbols/block} &= r leq m(d - 1) end{align} wheremis any positive integer. For many nonbinary BCH codes (called maximally redundant codes), the maximum number of check symbols per block is required, i.e.r = m(d - 1). Conditions whereby a primitive nonbinary BCH code is maximally redundant are discussed. It is shown that a class of codes exists, with symbols fromGF(q), based upon doubly lengthened Reed-Solomon codes overGF(q^m), having parameter values begin{align} text{block length} &= n = m(q^m + 1) \ text{check symbols/block} &= r = m(d - 1) \ text{designed distance} &= d end{align} where againmis any positive integer. Thus this class of codes extends the block length of maximally redundant codes by a multiplicative factor exceedingm, while retaining the same designed distance and same number of check symbols. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Fitting continuous probability density functions over [0, \infty ) using information theory ideas (Corresp.)

    Page(s): 226 - 230
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (728 KB)  

    The problem of estimating a probability density functionp(x)over[0, infty)when several low-order moments are known is considered. A computational procedure, based on information theory, is used to obtain the maximum entropy estimates ofp(x)in a number of cases. The situation when the first two moments only are known is considered in some detail. A table is included for estimatingp(x)when givenmu_1 ^{prime} , mu_2 ^{prime}withmu_2 ^{prime} leq 2(mu_1 ^{prime}) ^ 2. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Signal design for efficient detection in dispersive channels

    Page(s): 206 - 213
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1200 KB)  

    Signal design for maximizing the efficiency of the Neyman-Pearson detection procedure in randomly dispersive media is investigated. The medium is modeled as a randomly time-varying linear filter; by viewing the filter transfer function as a homogeneous random field on the time-frequency plane, a second-order theory results that relates various second-order measures of the time and frequency structures of input and output processes. A signal design strategy is developed that dictates transmitting signals that produce output processes with degrees of freedom possessing a signal-to-noise ratio (SNR) in the vicinity of 2. A distribution of signal energy in the output time-frequency plane that achieves the proper SNR for each degree of freedom is deduced and is used to infer constraints on input ambiguity functions that maximize detection efficiency. The general structure of efficient input signals for both high and low SNR is briefly discussed. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • On the representation of continuous parameter processes by a sequence of random variables

    Page(s): 139 - 141
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (448 KB)  

    This paper examines the question of representing a continuous parameter random process{ x_t, t in T }by a sequence of random variables "without loss of information." The principal result is that such a representation by expansion coefficients relative to a basis{ phi_i }ofmathcal{L}_2(T)is always possible, regardless of the orthogonality of{ phi_i }and of the boundedness of the time intervalT, provided only that the process is continuous in probability and almost every sample path has finite energy. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Further results on the synchronization of binary cyclic codes (Corresp.)

    Page(s): 238 - 241
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (560 KB)  

    It is shown that certain coset codes derived from binary cyclic codes can determine the magnitude of a synchronization error, as well as its direction by examining only the syndrome of the receivedntuple. For such coset codes, therefore, the need for a search procedure to recover synchronism is eliminated. In addition, the range of slip that can be detected and corrected for noisy channels is extended. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Alias-free randomly timed sampling of stochastic processes

    Page(s): 147 - 152
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1072 KB)  

    The notion of alias-free sampling is generalized to apply to random processesx(t)sampled at random timest_n; sampling is said to be alias free relative to a family of spectra if any spectrum of the family can be recovered by a linear operation on the correlation sequence{r(n)}, wherer(n) = E[x(l_{m+n}) overline{x(t_m)}]. The actual sampling timest_nneed not be known to effect recovery of the spectrum ofx(t). Various alternative criteria for verifying alias-free sampling are developed. It is then shown that any spectrum whatsoever can be recovered if{t_n}is a Poisson point process on the positive (or negative) half-axis. A second example of alias-free sampling is provided for spectra on a finite interval by periodic sampling (fort leq t_oort geq t_o) in which samples are randomly independently skipped (expunged), such that the average sampling rate is an arbitrarily small fraction of the Nyquist rate. A third example shows that randomly jittered sampling at the Nyquist rate is alias free. Certain related open questions are discussed. These concern the practical problems involved in estimating a spectrum from imperfectly known{ r(n) }. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Error probability bounds for systematic convolutional codes

    Page(s): 219 - 224
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (936 KB)  

    Upper and lower bounds on error probability for systematic convolutional codes are obtained. These bounds show that the probability of error for a systematic convolutional code is larger than the probability of error for a nonsystematic convolutional code of the same encoder-shift-register length. Two different interpretations of these results are given. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • The inversion of covariance matrices by finite Fourier transforms (Corresp.)

    Page(s): 230 - 232
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (400 KB)  

    A matrix series is derived that converges to the inverse of a covariance matrix. The members of the series are derived from a circular matrix that can be inverted by taking finite Fourier transforms. An example of the method is presented. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A note on filtering for independent increment processes (Corresp.)

    Page(s): 224 - 226
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (480 KB)  

    Using the equations of Fisher and Stear, a suboptimal filter for estimating the state of a linear system subject to non-Gaussian noise is developed. Some qualitative properties of this filter are derived. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Nonlinear estimation with quantized measurements--PCM, predictive quantization, and data compression

    Page(s): 152 - 161
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1344 KB)  

    Statistics conditioned on quantized measurements are considered in the general case. These results are specialized to Gaussian parameters and then extended to discrete-time linear systems. The conditional mean of the system's state vector may be found by passing the conditional mean of the measurement history through the Kalman filter that would be used had the measurements been linear. Repetitive use of Bayes' rule is not required. Because the implementation of this result requires lengthy numerical quadrature, two approximations are considered: the first is a power-series expansion of the probablity-density function; the second is a discrete-time version of a previously proposed algorithm that assumes the conditional distribution is normal. Both algorithms may be used with any memory length on stationary or nonstationary data. The two algorithms are applied to the noiseless-channel versions of the PCM, predictive quantization, and predictive-comparison data compression systems; ensemble-average performance estimates of the nonlinear filters are derived. Simulation results show that the performance estimates are quite accurate for most of the cases tested. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • The two-armed-bandit problem with time-invariant finite memory

    Page(s): 185 - 195
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1680 KB)  

    This paper solves the classical two-armed-bandit problem under the finite-memory constraint described below. Given are probability densitiesp_0andp_1, and two experimentsAandB. It is not known which density is associated with which experiment. Thus the experimental outcomeYof experimentAis as likely to be distributed according top_0as it is to be distributed according top_1. It is desired to sequentially choose an experiment to be performed on the basis of past observations according to the algorithmT_n = f(T_{n-1}, e_n, Y_n), e_n = e(T_{n-1}), whereT_n in {1, 2, cdots, m}is the state of memory at timen, e_n in {A, B}is the choice of experiment, andY_n, is the random variable observation. The goal is to maximize the asymptotic proportionrof uses of the experiment associated with densityp_0. Letl(y) = p_0 (y) / p_1 (y), and letbar{l}andbar{bar{l}}denote the almost everywhere greatest lower bound and least upper bound onl(y). Let1 = max {bar{bar{l}}, 1/bar{l}}. Then the optimal value ofr, over allm-state algorithms(f, e), will be shown to bel^{m-1} / (l^{m-1} + 1). Ane-optimal family ofm-state algorithms will be demonstrated. In general, optimal algorithms do not exist, ande-optimal algorithms require artificial randomization. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Bounds on performance of optimum quantizers

    Page(s): 172 - 184
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1632 KB)  

    A quantizerQdivides the range [0, 1] of a random variablexintoKquantizing intervals theith such interval having lengthDelta x_i. We define the quantization error for a particular value ofx(unusually) as the length of the quantizing interval in whichxfinds itself, and measure quantizer performance (unusually) by therth mean value of the quantizing interval lengthsM_r (Q) = overline{Delta x^{r^{1/r}}}, averaging with respect to the distribution functionFof the random variablex.Q_1is defined to be an optimum quantizer ifM_r (Q_1) leq M_r (Q)for allQ. The unusual definitions restrict the results to bounded random variables, but lead to general and precise results. We define a classQ^{ast}of quasi-optimum quantizers;Q_2is inQ^{ast}if the different intervalsDelta x_imake equal contributions to the meanrth power of the interval size so thatPr { Delta x_i } Delta x_{i^{r}}is constant for alli. Theorems 1, 2, 3, and 4 prove thatQ_2 in Q^{ast}exists and is unique for givenF, K, andr: that1 geq KM_r (Q_2) geq KM_r (Q_1) geq I_r, whereI_r = {int_0^{1} f (x)^p dx}^ {1/q}, fis the density of the absolutely continuous part of the distribution functionFofx, p = 1/(1+ r), andq = r /(1 + r): thatlim KM_r (Q_2) = I_rasK rightarrow infty; and that ifKM_r (Q) = I_rfor finiteK, thenQ=Q^{ast}. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • On the representation of a continuous stochastic intensity by Poisson shot noise

    Page(s): 142 - 147
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (968 KB)  

    In many applications a Poisson shot noise (PSN) process is said to statistically "represent" its intensity process. In this paper an investigation is made of the relationship between a PSN process and its intensity, when the latter is a sample function of a continuous stochastic process. The difference of the moments and the mean-square difference between the two processes are examined. The continuity assumption on the intensity permits the development of a sequence of moment relationships in which the effect of the PSN parameters can be seen. The results simplify and afford some degree of physical interpretation when the component functions of the PSN are "rectangular," or when the intensity process does not vary appreciably over their time width. An integral equation is derived that defines the component function that minimizes the mean-square difference between the two processes. It is shown that a "degenerate" form of component function induces complete statistical equality of the two processes. The problem has application to optical communication systems using photodetectors. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • An integral relevant to the detection of a signal in polarized Gaussian noise (Corresp.)

    Page(s): 232
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (136 KB)  

    An integral is calculated that arises while determining the probability of detecting a signal in polarized Gaussian noise and the probability of hitting a particular elliptical target with an imperfect weapon. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Flicker effect and television compression

    Page(s): 214 - 218
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (984 KB)  

    The perception of flicker by the human eye depends upon the picture content, and this paper describes a relation between the critical flicker frequencyf_cand the spatial frequencyf_x. Provided that the visual picture is presented in such a way that each spatial component, presented at a frequencyf_p, obeys the inequalityf_p geq f_c = f(f_x) = f_0 exp (-k f_x), no flicker effect is perceived. The constantsf_0andkare weakly dependent upon the brightness and picture area; they are a little more strongly dependent upon the contrast ratio and the type of picture. The effect can be exploited in motion pictures or television by presenting high-quality pictures interleaved with low-quality ones. In the latter, some high spatial frequencies would be omitted. For television, band reduction of a face by 9:1 is possible. Practical consideration may reduce this compression ratio to a little short of 2:1. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Full text access may be available. Click article title to sign in or learn about subscription options.
  • Information rates of Wiener processes

    Page(s): 134 - 139
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (800 KB)  

    Rate distortion functions are calculated for time discrete and time continuous Wiener processes with respect to the mean squared error criterion. In the time discrete case, we find the interesting result that, for0 leq D leq sigma^2 /4,R(D)for the Wiener process is identical toR(D)for the sequence of zero mean independent normally distributed increments of variance sigma^2 whose partial sums form the Wiener process. In the time continuous case, we derive the explicit formulaR(D) = 2 sigma^2 / ( pi^2 D), wheresigma^2is the variance of the increment daring a one-second interval. The resuitingR(D)curves are compared with the performance of an optimum integrating delta modulation system. Finally, by incorporating a delta modulation scheme in the random coding argument, we prove a source coding theorem that guarantees ourR(D)curves are physically significant for information transmission purposes even though Wiener processes are nonstationary. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.

Aims & Scope

IEEE Transactions on Information Theory publishes papers concerned with the transmission, processing, and utilization of information.

Full Aims & Scope

Meet Our Editors

Editor-in-Chief
Frank R. Kschischang

Department of Electrical and Computer Engineering