By Topic

Information Theory, IEEE Transactions on

Issue 1 • Date Jan 1986

Filter Results

Displaying Results 1 - 20 of 20
  • Adaptive buffer- instrumented entropy-coded quantizer performance for memoryless sources

    Page(s): 9 - 22
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1528 KB)  

    Variable-length codes can be used in entropy coding the outputs of an optimum entropy-constrained quantizer. Transmitting these codes over a synchronous channel; however, requires a buffer connecting the entropy coder to the channel. In a practical application, this buffer is of finite size and hence might overflow or undertow. To alleviate this difficulty, we use an adaptive scheme in which the quantizer parameters are changed successively according to the state of the buffer. Rate-distortion performance of optimum entropy-constrained quantizers in conjunction with this adaptive scheme is studied for the class of generalized Gaussian sources. It is demonstrated through simulations that the overflow/ undertow problem can be practically eliminated at the cost of a negligible increase in average distortion. Furthermore, it is shown that the efficiency of this system is more pronounced at high rates and for more broadtailed source densities. Easily computable upper and lower bounds on the average distortion of the adaptive system are developed. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Compression of two-dimensional data

    Page(s): 2 - 8
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (856 KB)  

    Distortion-free compressibility of individual pictures, i.e., two-dimensional arrays of data, by finite-state encoders is investigated. For every individual infinite pictureI, a quantityrho(I)is defined, called the compressibility ofI, which is shown to be the asymptotically attainable lower bound on the compression ratio that can be achieved forIby any finite-state information-lossless encoder. This is demonstrated by means of a constructive coding theorem and its converse that, apart from their asymptotic significance, might also provide useful criteria for finite and practical data-compression tasks. The proposed picture compressibility is also shown to possess the properties that one would expect and require of a suitably defined concept of two-dimensional entropy for arbitrary probabilistic ensembles of infinite pictures. While the definition ofrho(I)allows the use of different machines for different pictures, the constructive coding theorem leads to a universal compression scheme that is asymptotically optimal for every picture. The results are readily extendable to data arrays of any finite dimension. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Solving sparse linear equations over finite fields

    Page(s): 54 - 62
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1584 KB)  

    A "coordinate recurrence" method for solving sparse systems of linear equations over finite fields is described. The algorithms discussed all requireO(n_{1}(omega + n_{1})log^{k}n_{1})field operations, wheren_{1}is the maximum dimension of the coefficient matrix,omegais approximately the number of field operations required to apply the matrix to a test vector, and the value ofkdepends on the algorithm. A probabilistic algorithm is shown to exist for finding the determinant of a square matrix. Also, probabilistic algorithms are shown to exist for finding the minimum polynomial and rank with some arbitrarily small possibility of error. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • On the estimation of variance for autoregressive and moving average processes (Corresp.)

    Page(s): 120 - 125
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (848 KB)  

    The sample variance is commonly used to estimate the variance of stationary time series. When the second-order statistics of the process are known up to a scaling factor, this estimator is generally inefficient. In the case of an autoregressive (AR) process with unknown parameters, the sample variance is shown to be asymptotically efficient. However, the sample variance of a moving-average (MA) process with unknown parameters is generally an inefficient estimator. Closed-form expressions are derived for the Cramer-Rao hound associated with the variance estimation problem and for the variance of the sample-variance estimator, for both AR and MA processes. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Equidistant binary arithmetic codes (Corresp.)

    Page(s): 106 - 108
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (592 KB)  

    LetC(B)denote the binary cyclicANcode with generatorA, whereAB=2^{n} - 1. It is known thatC(B)is equidistant ifBis a prime powerp^{k}, where either2or-2is primitive moduloBprovidedpequiv 1 pmod{3}{rm if} k > 1. It is conjectured that these are the onlyBsuch thatC(B)is equidistant. We have verified this forB < 100 000. Several results are established that further limit the possibilities for counterexamples to the conjecture. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Memoryless detection of time-varying deterministic signals in dependent non-Gaussian noise (Corresp.)

    Page(s): 133 - 137
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (568 KB)  

    Memoryless discrete-time detection of a known time-varying signal in dependent non-Gaussian noise is considered. Optimal detectors are specified for cases where the detector nonlinearity is time-invariant and time-varying. A performance comparison is provided by way of examples. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Variance expressions for four spectral-centroid estimators of a complex Gaussian process

    Page(s): 73 - 78
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (752 KB)  

    Variance expressions are derived for four unbiased centroid estimators of an asymmetrical spectrum of the complex envelope of a narrow-band Gaussian process. These estimators are obtained by cross-correlating the undistorted or hard-clipped component and the time derivative of the undistorted or hard-dipped quadrature component of the complex envelope. When both components are hard-clipped, the variance expression of the centroid estimator is a generalization of Blachman's zero-crossing analysis. The variance of the centroid estimator obtained from finite records of the undistorted components is shown to be smaller than that of the other three estimators whenm_{2} > 5.32mu_{1}^{2}, wheremu_{1}andm_{2}are the first simple and second central spectral moments respectively. However, whenm_{2} < 5.32mu_{1}^{2}, the variance of the centroid estimate obtained from one hard-clipped component is shown to be smallest. Furthermore, the dependence of the variances on both the magnitude and direction of spectral asymmetry is shown to be different for the four centroid estimators considered. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A nonlocal approach for asymptotic memoryless detection theory (Corresp.)

    Page(s): 115 - 120
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1200 KB)  

    Previous literature on detector design has presented many tractable results using efficacy as a performance functional, which is a small signal or local approach. Some nonlocal issues of memoryless detection are considered. We argue that the asymptotic exponential rate of decrease of error probabilities, or Bayes risk, provides the appropriate asymptotic measure of performance. A main contribution is the development of an approximation for nonlocal design that retains the tractability of the efficacy approach. We have proved the local consistency of our approximation. Also, as a consequence of the analysis, new proofs for Pitman-Noether-like theorems have been obtained. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Efficient balanced codes

    Page(s): 51 - 53
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (432 KB)  

    Coding schemes in which each codeword contains equally many zeros and ones are constructed in such a way that they can be efficiently encoded and decoded. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • On the minimum distance of cyclic codes

    Page(s): 23 - 40
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (2176 KB)  

    The main result is a new lower bound for the minimum distance of cyclic codes that includes earlier bounds (i.e., BCH bound, HT bound, Roos bound). This bound is related to a second method for bounding the minimum distance of a cyclic code, which we call shifting. This method can be even stronger than the first one. For all binary cyclic codes of length< 63(with two exceptions), we show that our methods yield the true minimum distance. The two exceptions at the end of our list are a code and its even-weight subcode. We treat several examples of cyclic codes of lengthgeq 63. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Dual product constructions of Reed - Muller type codes (Corresp.)

    Page(s): 103 - 106
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (616 KB)  

    Various linear and nonlinearR(r,m)codes having parameters(2^{m}, 2^{k}, 2^{m-r})withk=sum_{i=0}^{r}left(^{m}_{i}right)are constructed fromR(r,q)andR(r,p)codes,m=p+q. A dual construction forR(m-r,m)codes fromR(p-r,p)andR(q-r,q)codes is also presented,m=p+q. As a simple corollary we have that the number of nonequivalentR(r,m)codes is at least exponential in the length (forr>1). ForR(m-r,m)codes, the lower bound is doubly exponential in the length (forr>1). View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Regular universal codeword sets (Corresp.)

    Page(s): 129 - 133
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (888 KB)  

    Proof is given that no regular or synchronizable universal codeword sets can achieve optimality. Some asymptotic properties of a remarkable class of regular codeword sets related to the Fibonacci representation of integers are also obtained. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Asymptotic error probability of binary hypothesis testing for Poisson point-process observations (Corresp.)

    Page(s): 113 - 115
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (440 KB)  

    It is shown that the asymptotic probability of error of a binary equiprobable hypothesis test for observed Poisson point processes with rateslambda_{i}(t)=b_{i}(t)+(rho_{i}(t)+z)^{2}, i=0,1, z rightarrow infty, is equal to the error probability of optimum deterministic-signal detection in additive white Gaussian noise when the signals coincide with the square roots of the point-process rates. The implication of this result in the error rate analysis of optical digital communication systems is discussed. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • On filtered binary processes

    Page(s): 63 - 72
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1392 KB)  

    The problem of calculating the probability density function of the output of anRCfilter driven by a binary random process with intervals generated by an equilibrium renewal process is studied. New integral equations, closely related to McFadden's original integral equations, are derived and solved by a matrix approximation method and by iteration. Transformations of the integral equations into differential equations are investigated and a new closed-form solution is obtained in one special case. Some numerical results that compare the matrix and iteration solutions with both exact solutions and approximate solutions based upon the Fokker-Planck equation are presented. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Soft decoding techniques for codes and lattices, including the Golay code and the Leech lattice

    Page(s): 41 - 50
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1520 KB)  

    Two kinds of algorithms are considered.1)If *** is a binary code of lengthn, a "soft decision" decoding algorithm for *** changes an arbitrary point ofR^{n}into a nearest codeword (nearest in Euclidean distance).2)Similarly, a decoding algorithm for a latticeLambdainR^{n}changes an arbitrary point ofR^{n}into a closest lattice point. Some general methods are given for constructing such algorithms, ami are used to obtain new and faster decoding algorithms for the Gosset latticeE_{8}, the Golay code the Leech lattice. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Optimum detection of a weak signal with minimal knowledge of dependency

    Page(s): 97 - 102
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (800 KB)  

    The optimum nonlinearity is defined for detection of a weak signal when minimal knowledge of the dependency structure of the observations is available. Specifically, it is assumed that the observations form a one-dependent strictly stationary sequence of random variables and that only a finite number of moments of the marginal density and the correlation coefficient between consecutive observations are known. It is assumed that the bivariate densities involved can be represented as diagonal series, using orthonormal polynomials. Using efficacy as a performance measure, the optimum nonlinearity is required to satisfy a saddle-point condition over this class of bivariate densities. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Binary DPSK transmission over terrestrial and satellite links (Corresp.)

    Page(s): 125 - 129
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (712 KB)  

    It has been observed previously that downlink noise correlation has no effect on the average error probability of a differentially phase shift keyed (DPSK) satellite system when the symbols are equiprobable. We show thai this observation holds for a satellite system exhibiting amplitude-modulation-to-amplitude-modulation (AM-AM) conversion effects and amplitude-modulation-to-phase-modulation (AM-PM) conversion effects. It is also shown thai noise correlation has no effect even when a constant phase error is caused by the delay line of the DPSK receiver in terrestrial and satellite links. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Minimum probability of error for asynchronous Gaussian multiple-access channels

    Page(s): 85 - 96
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1408 KB)  

    Consider a Gaussian multiple-access channel shared byKusers who transmit asynchronously independent data streams by modulating a set of assigned signal waveforms. The uncoded probability of error achievable by optimum multiuser detectors is investigated. It is shown that theK-user maximum-likelihood sequence detector consists of a bank of single-user matched filters followed by a Viterbi algorithm whose complexity per binary decision isO(2^{K}). The upper bound analysis of this detector follows an approach based on the decomposition of error sequences. The issues of convergence and tightness of the bounds are examined, and it is shown that the minimum multiuser error probability is equivalent in the Iow-noise region to that of a single-user system with reduced power. These results show that the proposed multiuser detectors afford important performance gains over conventional single-user systems, in which the signal constellation carries the entire burden of complexity required to achieve a given performance level. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • On nonparametric estimation of a functional of a probability density

    Page(s): 79 - 84
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (816 KB)  

    The nonparametric estimate derived from the Hermite orthogonal system of the functionalI=int f^{2}(x) dxwherefis an unknown probability density, is studied. Sufficient conditions for the weak and strong consistency of the estimate are presented, and the rate of convergence is given. In particular, under mild assumptions onf, the rate of mean-square error convergence isO(n^{-1}), whereas for almost complete convergence it isO((n^{-1} log n)^{1/2}). Moreover, several possible applications in the area of nonparametric inference of the estimate are indicated. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • State splitting for variable-length graphs (Corresp.)

    Page(s): 108 - 113
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (952 KB)  

    The state splitting algorithm of Adler, Coppersmith, and Hassner for graphs with edges of fixed length is extended to graphs with edges of variable lengths. This has the potential to improve modulation code construction techniques. Although the ideas of state splitting come from dynamical systems, completely graph-theoretic terms are used. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.

Aims & Scope

IEEE Transactions on Information Theory publishes papers concerned with the transmission, processing, and utilization of information.

Full Aims & Scope

Meet Our Editors

Editor-in-Chief
Frank R. Kschischang

Department of Electrical and Computer Engineering