By Topic

Information Theory, IEEE Transactions on

Issue 4 • Date July 1975

Filter Results

Displaying Results 1 - 25 of 34
  • Full text access may be available. Click article title to sign in or learn about subscription options.
  • Full text access may be available. Click article title to sign in or learn about subscription options.
  • Full text access may be available. Click article title to sign in or learn about subscription options.
  • Full text access may be available. Click article title to sign in or learn about subscription options.
  • Full text access may be available. Click article title to sign in or learn about subscription options.
  • Full text access may be available. Click article title to sign in or learn about subscription options.
  • The upper error bound of a new near-optimal code

    Page(s): 441 - 445
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1320 KB)  

    A code is implicitly constructed' from a lattice and its Dirichlet regions and, for Gaussian noise, the worst error probability of any code point is upperbounded in closed form by a chi-square distribution. The bound shows that fairly efficient codes can be obtained, particularly, at high signal-to-noise ratio (SNR) the bound approaches asymptotically the error bound of an optimal code. The derivation is by a promising new method in which the Minkowski-H!awka theorem of the geometry of numbers is used in place of the we!l-known random coding arguments. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • An achievable rate region for the broadcast channel

    Page(s): 399 - 404
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1520 KB)  

    Let p(y_1, y_2 \mid x) denote a discrete memoryless channel with a single source X and two independent receivers Y_1 and Y_2 . We exhibit an achievable region of rates (R_{11},R_{12},R_{22}) at which independent information can be sent, respectively, to receiver 1, to both receivers 1 and 2, and to receiver 2. The achievability of the region is shown by using a version of the asymptotic equipartition property involving many simultaneous "typicality" constraints. These results immediately generalize to yield an achievable rate region for the m -sender n -receiver channel in terms of standard mutual information quantities. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Tree encoding of speech

    Page(s): 379 - 387
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (2616 KB)  

    Recently developed methods of tree source coding with a fidelity criterion are applied to speech coding. We first demonstrate that tree codes are inherent in A-D speech convertors of the waveform following type and point to ordinary and adaptive delta modulation and differential pulse code modulation (DPCM) as examples. Insights of coding theory improve these trees at low rates; we offer two new code classes, one obtained by smoothing the DPCM tree and one using the rate-distortion theory of autoregressive sources. Using these codes, we study the performance of a simple synchronous tree searching algorithm called the M -algorithm that maintains a small fixed number of paths in contention. 1 and 2 bit/sample code trees, used to encode actual speech at 8, 10, and 16 kbits/s, yield improved dynamic range and channel error resistance, and 4-8 dB improvement in mean-square error (mse) over ordinary single-path searched DPCM. These improvements in excess of analytical estimates suggest that tree coding methods perform better with real-life sources than previously thought. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Generalized bandpass filters for decoding block codes

    Page(s): 417 - 422
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1760 KB)  

    We show that the optimum decoder for a linear code over a finite field can be implemented by a bank of bandpass filters that are defined in a generalized frequency domain. This decoder has a fixed signal processing structure and uses complex-valued arithmetic operations instead of the usual finite field ones, This structure closely resembles one of the usual digital filter mechanizations. The emphasis in this paper is on general additive channels. The maximum likelihood decoder configuration for a rate k/n code is a bank of k bandpass filters whose outputs are the decoded information digits. The parameters that define the filters are directly related to the channel statistics, and each of the band-pass filters is composed of r branch filters where the order of the field is the r th power of a prime. The processing requirements can further be reduced by detuning each branch filter and following it by a real roundoff operation. These possibilities have not been fully exploited and remain an open question. A procedure for determining the filter weights needed in the branch filters that uses fast transform techniques is demonstrated. The probability of correct decoding can be computed using functions defined over the transform domain. Finally, we show that suboptimal decoders, which may also include detection capabilities, can be derived from this work. A hierarchy of suboptimal decoding levels is proposed so as to reduce the average computational requirements of the decoder. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • On some properties of self-reciprocal polynomials (Corresp.)

    Page(s): 462 - 464
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (928 KB)  

    Self-reciprocal polynomials (SRP's) are used to generate reversible codes with a read backward property. We characterize all the irreducible SRP's of a given degree r . Furthermore, a method of obtaining a maximal exponent SRP is presented. A short table lists the maximal exponents of SRP's of degree up to r = 34 . View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Random coding strategies for minimum entropy

    Page(s): 388 - 391
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1080 KB)  

    This paper proves that there exists a fixed random coding Strategy for block coding a memoryless information source to achieve the absolute epsilon entropy of the source. That is, the strategy can be chosen independent of the block length. The principal new tool is an easy result on the semicontinuity of the relative entropy functional of one probability distribution with respect to another. The theorem generalizes a result from rate-distortion theory to the "zero-infinity" case. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Bounds for truncation error in sampling expansions of finite energy band-limited signals (Corresp.)

    Page(s): 482 - 485
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1032 KB)  

    An upper bound is established for the magnitude of the truncation error incurred when a real-valued finite energy signal that is bandlimited to -\pi r \leq \omega \leq \pi r, 0 < r < 1 , is approximated by 2N + 1 terms from its Shannon sampling series expansion with the samples taken at the integer points. The derived bound is an improvement on known results. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A nonlinear fixed-lag smoother for finite-state Markov processes

    Page(s): 446 - 452
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (2008 KB)  

    The fixed-lag smoothing of random telegraph type signals is studied. The smoothers are derived by first obtaining fixed-point smoothing equations and then using a time discretization. Simulation results are described that verify the qualitative carry-over of known results for the linear-Gaussian problem: the greater the lag, the greater the improvement; beyond a certain lag, no further improvement is obtained by the increase of lag; and the higher the signal-to-noise ratio (SNR), the greater is the improvement over filtering obtained through the use of smoothing. Smoothing errors of one-half the corresponding filtering error are demonstrated. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Autoregressive model fitting with noisy data by Akaike's information criterion (Corresp.)

    Page(s): 476 - 480
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1448 KB)  

    Davisson [131, [141 has considered the problem of determining the "order" of the signal from noisy data. Although interesting theoretically, his result is difficult to use in practice. In this correspondence, we exploit one well-known fact concerning autoregressive (AR) signals plus white noise, and using Akaike's information criterion [15], [17], we have developed one efficient procedure for determining the order of the AR signal from noisy data. The procedure is illustrated numerically using both artificially generated and real data. The connection between the preceding problem and the classical statistical problem of factor analysis is discussed. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A class of binoid single-error-correcting codes (Corresp.)

    Page(s): 468 - 469
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (576 KB)  

    A new class of group binoid single-error-correcting codes is given. The codes are nonbinary group codes over the additive group of integers modulo q . View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A class of nonparametric detectors for dependent input data

    Page(s): 431 - 437
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (2128 KB)  

    A class of nonparametric detectors is formulated for dependent input sequences of sampled data. Detectors in this class are nonadaptive modifications of standard nonparametric detectors designed for independent inputs and operate on multivariate samples derived by grouping the dependent univariate input samples. These groups are transformed to single variables and processed according to the corresponding standard nonparametric detection scheme. Performance analysis is initially based on assumptions implying independence of the multivariate samples, and results are then shown to remain valid under weaker conditions. Two modified one-channel detectors are analyzed to illustrate the improved performance achieved over corresponding standard non-parametric detectors. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Uniformly reasonable source encoding is often practically impossible

    Page(s): 368 - 373
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1816 KB)  

    The decoding of efficiently encoded messages, from either probabilistic, nonprobabilistic, or unknown message sources, is shown to be often practically impossible. If \tau (S) is a running-time bound on the computational effort of a decoder \Psi accepting a codeword P for message S , and \gamma [K_{\Psi }(S)] is an upper bound to acceptable codeword length \mid P \mid when the shortest codeword for S has length K_{\Psi }(S) , then for many message sources mathcal{M} there exist messages S \in mathcal{M} such that: 1) if the encoder satisfies \gamma , then the decoder violates \tau ; 2) if the decoder satisfies \tau , then the encoder violates \gamma . These conclusions remain valid even when we allow the decoder to reconstruct only an approximation S \prime in a neighborhood \delta (S) of S . The compatibility of these results with those of information theory rests upon the fact that we are inquiring into the detailed properties of coding systems for individual messages and not into the ensemble average properties. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • System error bounds for Lagrange polynomial estimation of band-limited functions (Corresp.)

    Page(s): 474 - 476
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (880 KB)  

    Several recent articles [1]-[3] have discussed the use of Lagrange polynomials for band-limited signal estimation. We find error bounds when using Lagrange polynomials for interpolation and extrapolation of finite-power band-limited signals if a finite number of regularly spaced noisy samples are used. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Hybrid low autocorrelation sequences (Corresp.)

    Page(s): 460 - 462
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (880 KB)  

    Skew-symmetric sequences of (2n + 1) terms, a_0,a_1,\cdots ,a_{2n} , are described for which the "merit factor" begin{equation} F_h = frac{biggl[sum_{i=0}^{2n} mid a_i mid biggr] ^2}{ 2 sum_{k=1}^{2n} biggl[ sum_{i=0}^{2n-k} text{sign} (a_i) cdot a_{i+k} biggl] ^2} end{equation} is unusually high. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Sliding-block source coding

    Page(s): 357 - 368
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (4032 KB)  

    Both noiseless source coding and source coding with a fidelity criterion are traditionally accomplished via the mapping of consecutive nonoverlapping source blocks into code blocks of fixed or variable length. Here we use an easy application and interpretation of the Kolmogorov-Ornstein isomorphism theorem of ergodic theory to prove the existence of a new class of noiseless source coding techniques consisting of nonlinear time-invariant discrete-time filters. The output codes are physically stationary, are not of variable length, require no buffers except for the filter memory, are not catastrophically affected by occasional channel errors, and provide a new interpretation of noiseless source coding. An information-theoretic interpretation of an early special case of the isomorphism theorem provides an example. The noiseless sliding-block theorem is then coupled with the sliding-block source coding subject to a fidelity criterion theorem to obtain a general sliding-block source coding theorem for noiseless and almost noiseless Channels. The approach, assumptions, and results are compared and contrasted with the special cases of quantization, delta modulation, and block stationary convolutional, trellis, tree, Viterbi, and sequential source coding techniques. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Nonparametric estimates of probability densities

    Page(s): 438 - 440
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (760 KB)  

    A disadvantage of the kernel estimate of a probability density is that the degree of smoothing about the observation points is chosen without regard to the data. A modification of the kernel estimate is proposed that allows the data to play a role in this smoothing and, at the same time, retains the desirable features of the kernel estimate. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • On the residual correlation of finite-dimensional discrete Fourier transforms of stationary signals (Corresp.)

    Page(s): 480 - 482
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (808 KB)  

    The covariance matrix of the Fourier coefficients of N - sampled stationary random signals is studied. Three theorems are established. 1) If the covariance sequence is summable, the magnitude of every off-diagonal covariance element converges to zero as N \rightarrow \infty . 2) If the covariance sequence is only square summable, the magnitude of the covariance elements sufficiently far from the diagonal converges to zero as N \rightarrow \infty . 3) If the covariance sequence is square summable, the weak norm of the matrix containing only the off-diagonal elements converges to zero as N \rightarrow \infty . The rates of convergence are also determined when the covariance sequence satisfies additional conditions. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Robustly optimal rate one-half binary convolutional codes (Corresp.)

    Page(s): 464 - 468
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1312 KB)  

    Three optimality criteria for convolutional codes are considered in this correspondence: namely, free distance, minimum distance, and distance profile. Here we report the results of computer searches for rate one-half binary convolutional codes that are "robustly optimal" in the sense of being optimal for one criterion and optimal or near-optimal for the other two criteria. Comparisons with previously known codes are made. The results of a computer simulation are reported to show the importance of the distance profile to computational performance with sequential decoding. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Maximum-likelihood syntactic decoding

    Page(s): 423 - 430
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (2392 KB)  

    A model of a linguistic information source is proposed as a grammar that generates a language over some finite alphabet. It is pointed out that grammatical sentences generated by the source grammar contain intrinsic "redundancy" that can be exploited for error-corrections. Symbols occurring in the sentences are composed according to some syntactic rules determined by the source grammar, and hence are different in nature from the lexicographical source symbols assumed in information theory and algebraic coding theory. Almost all programming languages and some simple natural languages can be described by the linguistic source model proposed in this paper. In order to combat excessive errors for very noisy channels, a conventional encoding-decoding scheme that does not utilize the source structure is introduced into the communication system. Decoded strings coming out of the lexicographical decoder may not be grammatical, which indicates that some uncorrected errors still remain in the individual sentences and will be reprocessed by a syntactic decoder that converts ungrammatical strings into legal sentences of the source language by the maximum-likelihood criterion. Thus more errors in the strings coming out of the noisy channel can be corrected by the syntactic decoder using syntactic analysis than the !exicographical decoder is capable of correcting or even of detecting. To design the syntactic decoder we use parsing techniques from the study of compilers and formal languages. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.

Aims & Scope

IEEE Transactions on Information Theory publishes papers concerned with the transmission, processing, and utilization of information.

Full Aims & Scope

Meet Our Editors

Editor-in-Chief
Frank R. Kschischang

Department of Electrical and Computer Engineering