By Topic

Information Theory, IEEE Transactions on

Issue 4 • Date July 1986

Filter Results

Displaying Results 1 - 19 of 19
  • Comments, with reply, on "ARMA spectral estimation of time series with missing observations" by B. Porat and B. Friedlander

    Page(s): 601 - 602
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (255 KB)  

    A variant to a recently proposed autoregressive moving average (ARMA) spectrum estimation technique for time series with gapped data is suggested. It is based on the partial fraction expansion of the power spectrum and exhibits some computational and operational advantages. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Analysis of a delayed delta modulator

    Page(s): 496 - 512
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (2136 KB)  

    While delta modulation (DM) simply compares the current predictive estimate of the input with the current sample, delayed delta modulation (DDM) also compares with the upcoming sample so as to detect and anticipate slope overloading. Since this future sample must be available before the present output is determined and the estimate updated, delay is introduced at the encoding. The performance of DDM with perfect integration and step-function reconstruction is analyzed for each of three random input signals. In every case, the stochastic stability of the system is established. For a discrete time, independent and identically distributed input, the (limiting) joint distribution of input and output is derived, and the (asymptotic) mean-square sample point error mse(SP) is computed when the input is Gaussian. For a Wiener input, the joint distribution of the sample point and prediction errors is derived, and mse(SP) and the time-averaged mse (mse(TA)) are computed. For a stationary first-order Gauss-Markov input, the joint distribution of input and output is derived and mse(SP) and mse(TA) computed. Graphs of the mse's illustrate the improvement attainable by using DDM instead of DM. With optimal setting of parameters, mse(SP) (mse(TA)) is reduced about 15 percent ( 35 percent). View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Characterizing partition detectors with stationary and quasi-stationary Markov dependent data

    Page(s): 471 - 482
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1672 KB)  

    Multisensor data from sonar arrays are usually temporally and spatially dependent. In addition, in some environments the noise field in which the sonar system must operate is contaminated by impulsive and non-Gaussian interference. A natural way to include these effects is to formulate a statistic based upon the likelihood ratio. However, implementing a multisensor likelihood ratio statistic in its full generality is a formidable task. Our approach to solving the implementation complexity problem is to partition the amplitude of each sensor output into m intervals and model the resulting data as a finite Markov chain. Certain performance measures for fixed-sample-size and sequential detection statistics based on Markov chains are considered. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • An algorithm for complex approximations in Z[e^{2{\pi}i/8}] (Corresp.)

    Page(s): 603 - 607
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (912 KB)  

    An algorithm is described that approximates complex numbers by elements of the algebraic integers of Z[e^{2 \pi i / 8}] with integer coordinates of at most a prescribed size. The motivating application is to reduce the dynamic range requirements of residue number system implementations of the discrete Fourier transform. The closest points to zero of Z[e^{2 \pi i / 8}]_{M} gor any integer M are determined. A particular sequence of such points forms the basis of the algorithm. An example of 8 -bit Z[\omega ]_{M} - approximations of the 128th roots of unity is considered. The algorithm yields M = 186; with scaling M is reduced to 18 . View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Complexity of strings in the class of Markov sources

    Page(s): 526 - 532
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1096 KB)  

    Shannon's self-information of a string is generalized to its complexity relative to the class of finite-state-machine (FSM) defined sources. Unlike an earlier generalization, the new one is valid for both short and long strings. The definition is justified in part by a theorem stating that, asymptotically, the mean complexity provides a tight lower bound for the mean length of all so-called regular codes. This also generalizes Shannon's noiseless coding theorem. For a large subclass of FSM sources a simple algorithm is described for computing the complexity. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Smoothing error dynamics and their use in the solution of smoothing and mapping problems

    Page(s): 483 - 495
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1896 KB)  

    Martingale decomposition techniques are used to derive Markovian models for the error in smoothed estimates of processes described by linear models driven by white noise. These models, together with some simple Hilbert space decomposition ideas, provide a simple unified framework for examining a variety of problems involving the efficient assimilation of spatial data, which we refer to as mapping problems. Algorithms for several different mapping problems are derived. A specific example of map updating for a two-dimensional random field is included. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Distribution of the filtered output of a quadratic rectifier computed by numerical contour integration

    Page(s): 450 - 463
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1496 KB)  

    The cumulative distribution of the filtered output of a quadratic rectifier whose input is either narrow-band Gaussian noise or Gaussian noise with a low-pass spectral density is to be computed by numerical quadrature of a Laplace inversion integral along a contour in the complex plane chosen to economize the number of steps. The integrand contains the moment-generating function (mgf) of the output. It is expressed in terms of the Fredholm determinant and the resolvent kernel associated with an integral equation involving the autocovariance function of the input and the impulse response of the output filter. A special case is the power of a mean-zero Gaussian process averaged over a finite interval, and when this process has a rational spectral density, the mgf can be expressed as the ratio of certain finite determinants. By this method distributions are calculated for low-pass noise with RLC and second- and fourth-order Chebyshev spectral densities. For rational input spectral densities but arbitrary positive output filtering and an arbitrary additive input signal, the mgf can be calculated by integrating differential equations of the Kalman-Bucy type. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • The complexity of information extraction

    Page(s): 513 - 525
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (2312 KB)  

    How difficult are decision problems based on natural data, such as pattern recognition? To answer this question, decision problems are characterized by introducing four measures defined on a Boolean function f of N variables: the implementation cost C(f) , the randomness R(f) , the deterministic entropy H(f) , and the complexity K(f) . The highlights and main results are roughly as follows, l) C(f) \approx R(f) H(f) \approx K(f) , all measured in bits. 2) Decision problems based on natural data are partially random (in the Kolmogorov sense) and have low entropy with respect to their dimensionality, and the relations between the four measures translate to lower and upper bounds on the cost of solving these problems. 3) Allowing small errors in the implementation of f saves a lot in the iow entropy case but saves nothing in the high-entropy case. If f is partially structured, the implementation cost is reduced substantially. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Decoding the Golay codes

    Page(s): 561 - 567
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1064 KB)  

    We introduce exceptionally simple decoding algorithms for the two extended Golay codes. The algorithms are based on recent methods of Conway and Curtis of finding the unique blocks containing five points in either the (5,8,24) Steiner system or the (5,6,12) Steiner system. These decoding methods are simple enough to enable decoding extended Golay codes by hand. Both of the methods involve relations between the extended Golay codes and other self-dual codes. Proofs are given explaining these relationships and why the decoding methods work. The decoding algorithms are explained and illustrated with many examples. [3 , chap. 12] has facts about the Mathieu group and some details about decoding the Golay codes. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • The Gaussian Nonlinearity's Output Power Spectral Density Due to a Sinusoid Plus Band-Limited Noise

    Page(s): 597 - 601
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (520 KB)  

    A sinusoid accompanied by stationary, additive, nonzero-mean, band-limited, white Gaussian noise is passed through a Gaussian nonlinear characteristic, and the output power spectral density is evaluated by using known results due to Rice and Atherton. Several characteristics of the Gaussian nonlinearity are revealed in the process, and input tuning is shown to contribute to output noise reduction. The results are applicable in the analysis of tracking systems employing sinusoidal dithers. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Extrapolating a band-limited function from its samples taken in a finite interval

    Page(s): 464 - 470
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1056 KB)  

    A band-limited signal of finite energy can be reconstructed from its samples taken at the Nyquist rate. Moreover, the reconstruction is stable, a feature crucial for implementation: a small error in the sample values generates only a correspondingly small error in the resulting signal. The Nyquist sample values are mutually independent, so that knowledge of them in a given interval - T \leq t \leq 0 provides hardly any information about the behavior of the signal outside the interval. However, when the samples are taken at a greater rate--a case referred to as "over-sampling"--they are interrelated, and this redundancy can be exploited in various ways to improve the behavior of the reconstruction procedure. A natural question is whether it can also be used to form accurate estimates of the signal outside the interval of observation; this problem is relevant as well to prediction theory. With oversampling, when T = \infty , so that the samples are known on the entire half-line t < 0 , they determine the signal everywhere, although the reconstruction is now no longer stable. Here we examine the case of finite T; of course, a finite amount of data can yield only limited accuracy. We prove that the samples can be used to form an approximation to the signal outside the sampling interval, with an error which, asymptotically as T \rightarrow \infty , decreases exponentially in T , over a range which grows linearly with T . However, as in the limiting case, these approximations are not useful in practice, since they require the sample values to be known exactly. In the presence of measurement error, the nature of the results changes: good approximations are now available for only a bounded distance outside the interval of observation, regardless of its length, and their accuracy and range of validity can be increased only by improving the precision of sample reading. Since physical measurements are never perfect, it is this conclusion which counts for applications. The same results hold for the extrapolation of bounded band-limited signals. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Hypothesis testing with communication constraints

    Page(s): 533 - 542
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1384 KB)  

    A new class of statistical problems is introduced, involving the presence of communication constraints on remotely collected data. Bivariate hypothesis testing, H_{0}: P_{XY} against H_{1}: P_{={XY}} , is considered when the statistician has direct access to Y data but can be informed about X data only at a preseribed finite rate R . For any fixed R the smallest achievable probability of an error of type 2 with the probability of an error of type 1 being at most \epsilon is shown to go to zero with an exponential rate not depending on \epsilon as the sample size goes to infinity. A single-letter formula for the exponent is given when P_{={XY}} = P_{X} \times P_{Y} (test against independence), and partial results are obtained for general P_{={XY}} . An application to a search problem of Chernoff is also given. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • On multiple descriptions and team guessing

    Page(s): 543 - 549
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (936 KB)  

    Witsenhausen's hyperbola bound for the multiple description problem without excess rate in case of a binary source is not tight for exact joint reproductions. However, this bound is tight for almost-exact joint reproductions (Theorem 1 , conjectured by Witsenhausen). The proof is based on an {em approximative} form of the team guessing lemma for {em sequences} of random variables. (This result may be of interest also for team guessing). The hyperbola bound is also tight for exact joint reproductions and arbitrarily small, but positive, excess rate (Theorem 2) . The proof of this result uses our covering lemma. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A continuous polling system with constant service times

    Page(s): 584 - 591
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1080 KB)  

    A server moves at a constant rate around a closed tour, stopping to perform services wherever requests are encountered. Requests appear according to a Poisson process at locations independently and uniformly distributed over the tour, and the service times are all taken to be one unit. Discrete versions of this model have applications ranging from machine repair to computer/communication polling systems. The distributions of the number served in a cycle and the number of waiting requests and their waiting times are calculated, and defections are studied. Because our continuous model is simpler than the discrete models, more easily interpreted formulas and some results that have yet to be obtained for discrete models are acquired. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • An iterative code construction and codes generated by permutations for the asymmetric multiple-access channel (Corresp.)

    Page(s): 607 - 612
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1496 KB)  

    It is shown that for an asymmetric multiple-access channel, it is possible, starting with one codeword at each of the encoders, to add the codewords for one message at a time (first for the elements in the set of private messages and thereafter for the messages in the set of common messages) and to construct this way, iteratively, a code for this channel with the average error probability upper-bounded by an exponentially small quantity. Using this result, it is subsequently shown that a code for the asymmetric multiple-access channel can also be generated by only. a few (linear in the block length) permutations, still keeping the average error probability upper-bounded by the same error bound and in this way gaining storage space. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Robust coding for multiple-access channels

    Page(s): 550 - 560
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1720 KB)  

    The problem of minimax robust coding for classes of multiple-access channels with uncertainty in their statistical description is addressed. We consider 1) discrete memoryless multiple-access channels with uncertainty in the probability transition matrices and 2) discrete-time stationary additive Gaussian multiple-access channels with spectral uncertainty. The uncertainty is modeled using classes determined by two-alternating Choquet capacities. Both block codes and tree codes are considered. A robust maximum-likelihood decoding rule is derived which guarantees that, for ali two-user channels in the uncertainty class and all pairs of code rates in a critical rate region, the average probability of decoding error for the ensemble of pairs of random block codes and the ensemble of pairs of random tree codes converges to zero exponentially with increasing block length or constraint length, respectively. The channel capacity and cutoff rate regions of the class are then evaluated. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A pyramid vector quantizer

    Page(s): 568 - 583
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1784 KB)  

    The geometric properties of a memoryless Laplacian source are presented and used to establish a source coding theorem. Motivated by this geometric structure, a pyramid vector quantizer (PVQ) is developed for arbitrary vector dimension. The PVQ is based on the cubic lattice points that lie on the surface of an L -dimensional pyramid and has simple encoding and decoding algorithms. A product code version of the PVQ is developed and generalized to apply to a variety of sources. Analytical expressions are derived for the PVQ mean square error (mse), and simulation results are presented for PVQ encoding of several memoryless sources. For large rate and dimension, PVQ encoding of memoryless Laplacian, gamma, and Gaussian sources provides rose improvements of 5.64, 8.40 , and 2.39 dB, respectively, over the corresponding optimum scalar quantizer. Although suboptimum in a rate-distortion sense, because the PVQ can encode large-dimensional vectors, it offers significant reduction in rose distortion compared with the optimum Lloyd-Max scalar quantizer, and provides an attractive alternative to currently available vector quantizers. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Ungerboeck codes do not shape the signal power spectrum (Corresp.)

    Page(s): 595 - 596
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (288 KB)  

    Simple sufficient conditions are found for the spectrum of Ungerboeck-coded signals to be equal to the spectrum of uncoded signals. It is shown that a carefully designed Ungerboeck code does not shape the spectrum. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • An achievable bound for optimal noiseless coding of a random variable (Corresp.)

    Page(s): 592 - 594
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (376 KB)  

    For a discrete N -valued random variable ( N possibly denumerably infinite) Leung-Yan-Cheong and Cover have given bounds for the minimal expected length of a one-to-one (not necessarily uniquely decodable) code L_{1:1}=\sum _{i=1}^{N} p_{i} \log \left( frac{1}{2} + 1 \right). It is shown that the best possible case occurs for certain denumerably infinite sets of nonzero probabilities. This absolute bound is related to the Shannon entropy H of the distribution by (h (\cdot) is the binary entropy function). View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.

Aims & Scope

IEEE Transactions on Information Theory publishes papers concerned with the transmission, processing, and utilization of information.

Full Aims & Scope

Meet Our Editors

Editor-in-Chief
Frank R. Kschischang

Department of Electrical and Computer Engineering