Notification:
We are currently experiencing intermittent issues impacting performance. We apologize for the inconvenience.
By Topic

Information Theory, IEEE Transactions on

Issue 5 • Date September 1979

Filter Results

Displaying Results 1 - 20 of 20
  • Correction to 'Kalman Filtering in Two Dimensions'

    Publication Year: 1979 , Page(s): 628 - 629
    Cited by:  Papers (6)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (257 KB)  

    In the above paper, ^{1} the two-dimensional (2-D) reduced update Kalman filter was introduced as an optimal approximation to the 2-D Kalman filter for scalar observations obtained from a raster scan. The reduced update filter consists of two parts: a prediction part and a reduced update part, i.e., an update of View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Review of 'Digital Image Processing' (Gonzalez, R. C., and Wintz, P.; 1977)

    Publication Year: 1979 , Page(s): 630 - 631
    Save to Project icon | Request Permissions | PDF file iconPDF (336 KB)  
    Freely Available from IEEE
  • Access-control disciplines for multi-access communication channels: Reservation and TDMA schemes

    Publication Year: 1979 , Page(s): 516 - 536
    Cited by:  Papers (53)  |  Patents (1)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (3416 KB)  

    Reservation and TDMA schemes are studied for governing the access-control discipline for a network of terminals communicating through a multi-access broadcast channel. A single repeater is employed to allow a fully connected network structure. A channel can be characterized as inducing a Iow propagation-delay value, as for terrestrial radio or fine networks, or as being associated with a higher propagation-delay value, as for a satellite communication channel. A synchronized (slotted) communication medium is considered. Messages are composed of a random number of packets, governed by an arbitrary message-length distribution. The process describing the number of reserved message arrivals within each time frame is assumed to be a sequence of i.i.d, random variables, governed by an arbitrary distribution. (A Poisson arrival stream thus becomes a special case.) The reservation access-control disciplines studied in this paper employ message-switching distributed-control procedures. The performance of each access-control scheme is evaluated according to its delay-throughput function. In particular, schemes are developed to adapt their structure, or protocol, dynamically to the underlying fluctuating network traffic-flow values. A fixed-reservation access-control (FRAC) discipline is studied, employing a fixed periodic pattern of reservation and service periods. The reservation periods are used for the transmission of reservation packets as well as for the integrated service of other groups of network stations. The latter stations can access the channel during these periods, using any proper access control procedure. As a special case, message-delay distributions and moments under a TDMA scheme are obtained. Using dynamic estimates of the underlying message traffic parameters, a dynamic fixed-reservation access-control (DFRAC) scheme is obtained. An analytical technique, which employs a Markov ratio limit theorem, is presented for the derivation of the delay-throughput performance curves of dynamic demand-assignment reservation schemes. To illustrate its application, asynchronous reservation demand-assignment (ARDA) schemes are developed to adapt automatically to the underlying network traffic characteristics. Such schemes establish reservation slots dynamically acc- ording to observed network service demands and queue sizes. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Quantization schemes for bivariate Gaussian random variables

    Publication Year: 1979 , Page(s): 537 - 543
    Cited by:  Papers (13)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (792 KB)  

    The problem of quantizing two-dimensional Gaussian random variables is considered. It is shown that, for all but a finite number of cases, a polar representation gives a smaller mean square quantization error than a Cartesian representation. Applications of the results to a transform coding scheme known as spectral phase coding are discussed. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Universal source coding of finite alphabet sources via composition classes

    Publication Year: 1979 , Page(s): 549 - 557
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1384 KB)  

    The universal source coding theorem for stationary sources states that, by using a preselected code with sufficiently large block length, any stationary source can be encoded arbitrarily close to the optimum performance theoretically attainable for the source. The code construction typically used to prove this theorem offers no performance guarantees for nonstationary sources. A code construction (based on composition classes) is described which is shown to be universal for stationary sources. In addition, this code construction guarantees that the distortion for any block in the sequence will not exceed a maximum value determined by its composition. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • On the algorithmic foundation of information theory

    Publication Year: 1979 , Page(s): 557 - 566
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1616 KB)  

    The information content of binary sequences is defined by minimal program complexity measures and is related to computable martingales. The equivalence of the complexity approach and the martingale approach after restriction to effective random tests is used to establish generalized source coding theorems and converses. Finite state complexity and decomposable martingales are related to classical block codes and the relative frequency behavior of sequences. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Detection of weak signals in narrowband non-Gaussian noise

    Publication Year: 1979 , Page(s): 592 - 600
    Cited by:  Papers (24)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1176 KB)  

    A class of nonlinear receiver structures is described for the detection of weak signals in non-Gaussian narrowband noise. In particular, the concept of a locally optimum receiver structure is extended to the ease of narrowband signal and noise models. A useful class of non-Gaussian narrowband noise models is developed for which the locally optimum receiver implementation is explicitly determined. These structures are shown to provide considerable improvement over conventional linear receiver structures. The basis of comparison is taken as the asymptotic relative efficiency (ARE). Unfortunately, the locally optimum receiver requires explicit {\sl a priori} knowledge of the underlying noise distribution. To circumvent this difficulty a rather simple adaptive nonlinear receiver structure is described which attempts to adapt to the unknown prevailing noise environment. This adaptive receiver is shown to provide fairly efficient and robust performance in a wide variety of non-Gaussian narrowband noise environments. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Tree algorithms for packet broadcast channels

    Publication Year: 1979 , Page(s): 505 - 515
    Cited by:  Papers (348)  |  Patents (50)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1472 KB)  

    The multiaccessing of a broadcast communication channel by independent sources is considered. Previous accessing techniques suffer from long message delays, low throughput, and/or congestion instabilities. A new class of high-speed, high-throughput, stable, multiaccessing algorithms is presented. Contentions resolving tree algorithms are introduced, and they are analyzed for specific probabilistic source models. It is shown that these algorithms are stable (in that all moments of delay exist) and are optimal in a certain sense. Furthermore, they have a maximum throughput of 0.430 packets/slut and have good delay properties. It is also shown that, under heavy traffic, the optimally controlled tree algorithm adaptively changes to the conventional time-division multiple access protocol. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Symmetries of binary Goppa codes (Corresp.)

    Publication Year: 1979 , Page(s): 609 - 612
    Cited by:  Papers (1)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (752 KB)  

    It is known that extended Goppa cedes are invariant under the group of transformations Z \rightarrow (A Z + B ) / ( CZ + D ) , with A D + BC \neq 0 . This invariance is used here to classify cubic and quartic irreducible Goppa codes and to investigate their symmetry groups. A computer has been used to determine the actual group of the codes of length 33 (for cubics and quarries). It has been said, concerning the trends in symmetry groups with respect to the Gilbert bound, that "a good family of codes can be linear or have many symmetries, hut not both" [8]. The groups found here are rather small; and so the results reinforce that statement. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Computational moments for sequential decoding of convolutional codes

    Publication Year: 1979 , Page(s): 584 - 591
    Cited by:  Papers (5)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (944 KB)  

    The long standing conjecture is established that, for a discrete memoryless channel, there exists a linear convolutional code with infinite constraint length such that the \rho th (\rho \geq 1) moment of the number of F -hypotheses in the Fano sequential decoding algorithm is bounded, provided that the transmission rate R is less than E_{0}( \rho,r)/ \rho , where r(x) is a distribution over the channel input alphabet. A new concept of independence for a finite set of message sequences plays an essential role in averaging a product of likelihood ratios over an ensemble of code sequences in a code tree. A simpler version of the method can be applied to the proof of the conjecture for general tree codes. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • The source coding theorem revisited: A combinatorial approach

    Publication Year: 1979 , Page(s): 544 - 548
    Cited by:  Papers (11)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (752 KB)  

    A combinatorial approach is proposed for proving the classical source coding theorems for a finite memoryless stationary source (giving achievable rates and the error probability exponent). This approach provides a sound heuristic justification for the widespread appearence of entropy and divergence (Kullback's discrimination) in source coding. The results are based on the notion of composition class -- a set made up of all the distinct source sequences of a given length which are permutations of one another. The asymptotic growth rate of any composition class is precisely an entropy. For a finite memoryless constant source all members of a composition class have equal probability; the probability of any given class therefore is equal to the number of sequences in the class times the probability of an individual sequence in the class. The number of different composition classes is algebraic in block length, whereas the probability of a composition class is exponential, and the probability exponent is a divergence. Thus if a codeword is assigned to all sequences whose composition classes have rate less than some rate R , the probability of error is asymptotically the probability of the must probable composition class of rate greater than R . This is expressed in terms of a divergence. No use is made either of the law of large numbers or of Chebyshev's inequality. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • The weighted nearest neighbor rule for class dependent sample sizes (Corresp.)

    Publication Year: 1979 , Page(s): 617 - 619
    Cited by:  Papers (10)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (384 KB)  

    The nearest neighbor rule is considered for data samples obtained by selecting N_{i} independent samples with the conditional distribution corresponding to class C_{i} . It is shown that a weighted distance rule can The nearest neighbor nde is considered for data samples obtained by selecting N_{i} independent samples with the conditional distribution corresponding to class C_{i} . It is shown that a weighted distance rule can improve the performance when the ratio of N_{i} to the total sample size differs substantially from the {\sl a priori} probability of class C_{i} . View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Distribution-free performance bounds for potential function rules

    Publication Year: 1979 , Page(s): 601 - 604
    Cited by:  Papers (9)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (440 KB)  

    In the discrimination problem the random variable \theta , known to take values in {1, \cdots ,M} , is estimated from the random vector X . All that is known about the joint distribution of (X, \theta) is that which can be inferred from a sample (X_{1}, \theta_{1}), \cdots ,(X_{n}, \theta_{n}) of size n drawn from that distribution. A discrimination nde is any procedure which determines a decision \hat{ \theta} for \theta from X and (X_{1}, \theta_{1}) , \cdots , (X_{n}, \theta_{n}) . For rules which are determined by potential functions it is shown that the mean-square difference between the probability of error for the nde and its deleted estimate is bounded by A/ \sqrt {n} where A is an explicitly given constant depending only on M and the potential function. The O(n ^{-1/2}) behavior is shown to be the best possible for one of the most commonly encountered rules of this type. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • On the strong information singularity of certain stationary processes (Corresp.)

    Publication Year: 1979 , Page(s): 605 - 609
    Cited by:  Papers (1)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (840 KB)  

    In an exploratory paper, T. Berger studied discrete random processes which generate information slower than linearly with time. One of his objectives was to provide a physically meaningful definition of a deterministic process, and to this end he introduced the notion of strong information singularity. His work is supplemented by demonstrating that a large class of convariance stationary processes are strongly information singular with respect to a class of stationary Gaussian processes. One important consequence is that for a large class of covariance stationary processes the information rate equals that of the process associated with the Brownian motion component of the spectral representation. In an exploratory paper, T. Berger studied discrete random processes which generate information slower than linearly with time. One of his objectives was to provide a physically meaningful definition of a deterministic process, and to this end he introduced the notion of strong information singularity. His work is supplemented by demonstrating that a large class of convariance stationary processes are strongly information singular with respect to a class of stationary Gaussian processes. One important consequence is that for a large class of covariance stationary processes the information rate equals that of the process associated with the Brownian motion component of the spectral representation. In an exploratory paper, T. Berger studied discrete random In an exploratory paper, T. Berger studied discrete random processes which generate information slower than linearly with time. One of his objectives was to provide a physically meaningful definition of a deterministic process, and to this end he introduced the notion of strong information singularity. His work is supplemented by demonstrating that a large class of convariance stationary processes are strongly information singular with respect to a class of stationary Gaussian processes. One important consequence is that for a large class of covariance stationary processes the information rate equals that of the process associated with the Brownian motion component of the spectral representation. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • The average number of weighings to locate a counterfeit coin (Corresp.)

    Publication Year: 1979 , Page(s): 616 - 617
    Cited by:  Papers (1)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (368 KB)  

    A solution is obtained to the problem of finding the minimum of the average number of weighings, using a balance, needed to locate one light coin from among n coins of which n - 1 are genuine and of the same weight. Optimal strategies are determined, and the anomaly that it may be better to work with n + 1 coins rather than n is examined and explained. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Capacity theorems for the relay channel

    Publication Year: 1979 , Page(s): 572 - 584
    Cited by:  Papers (1340)  |  Patents (16)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1744 KB)  

    A relay channel consists of an input x_{l} , a relay output y_{1} , a channel output y , and a relay sender x_{2} (whose transmission is allowed to depend on the past symbols y_{1} . The dependence of the received symbols upon the inputs is given by p(y,y_{1}|x_{1},x_{2}) . The channel is assumed to be memoryless. In this paper the following capacity theorems are proved. 1)If y is a degraded form of y_{1} , then C : = : \max !_{p(x_{1},x_{2})} \min ,{I(X_{1},X_{2};Y), I(X_{1}; Y_{1}|X_{2})} . 2)If y_{1} is a degraded form of y , then C : = : \max !_{p(x_{1})} \max _{x_{2}} I(X_{1};Y|x_{2}) . 3)If p(y,y_{1}|x_{1},x_{2}) is an arbitrary relay channel with feedback from (y,y_{1}) to both x_{1} and x_{2} , then C: = : \max _{p(x_{1},x_{2})} \min ,{I(X_{1},X_{2};Y),I ,(X_{1};Y,Y_{1}|X_{2})} . 4)For a general relay channel, C \leq \hbox{max}_{p(x_{1},x_{2})} \hbox{ min} \{ I(X_{1}, X_{2};Y),I(X_{1};Y,Y_{1}|X_{2}) . Superposition block Markov encoding is used to show achievability of C , and converses are established. The capacities of the Gaussian relay channel and certain discrete relay channels are evaluated. Finally, an achievable lower bound to the capacity of the general relay channel is established. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Detection of a coherent quantum signal in thermal noise (Corresp.)

    Publication Year: 1979 , Page(s): 624 - 628
    Cited by:  Papers (3)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (744 KB)  

    Methods are presented for calculating the minimum attainable probability of error in detecting a coherent quantum signal received amid thermal noise in the limits of Iow and high signal-to-noise ratio. In the latter limit, quantum-mechanical perturbation theory is applied to solving the detection operator equation approximately. Graphical results are furnished for binary signals transmitted with equal prior probabilities. Previous work on this problem is thus extended to a broader range of signal and noise strengths. The results are applied to the detection of antipodal two-photon-coherent-state signals. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Optimum linear coding in continuous-time communication systems with noisy side information at the decoder (Corresp.)

    Publication Year: 1979 , Page(s): 612 - 616
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (752 KB)  

    The problem of optimal coding in a communication system with a noiseless feedback link and with noisy side information at the decoder is treated. The message is taken as a Gaussian random variable and both the main and the side channels are assumed to be continuous white-Gaussian. A linear encoding-decoding scheme is developed which attains the well-known performance bounds for the special cases of i) noiseless side information and ii) no side information. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Higher-dimensional Hadamard matrices

    Publication Year: 1979 , Page(s): 566 - 572
    Cited by:  Papers (5)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1072 KB)  

    The concept of a Hadamard matrix as a binary orthogonal matrix is extended to higher dimensions. An n -dimensional Hadamard matrix [h_{ijk \cdots n}] is defined as one in which all parallel (n - 1) -dimensional layers, in any axis-normal orientation, are uncorrelated. This is equivalent to the requirements that h_{ijk \cdots n} = \pm1 and that \sum _{p} \sum _{q} \sum _{r} \cdots \sum _{y} h_{pqr \cdots yb}= m^{(n-1)} \delta _{ab} where (pqr \cdots yz) represents all permutations of (ijk \cdots n) . A "proper" n -dimensional Hadamard matrix is defined as a special case of the above in which all two-dimensional layers, in all axis-normal orientations, are Hadamard matrices, as a consequence of which all intermediate-dimensional layers are also Hadamard matrices. Procedures are described for deriving three- and four-dimensional Hadamard matrices of varying propriety from two-dimensional Hadamard matrices. A formula is given for a fully proper n -dimensional matrix of order two, which can be expanded by direct multiplication to yield proper (2^{t})^{n} Hadamard matrices. It is suggested that proper higher dimensional Hadamard matrices may find application in error-correcting cedes, where their hierarchy of orthogonalitias permit a variety of checking procedures. Other types of Hadamard matrices may be of use in security codes on the basis of their resemblance to random binary matrices. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Point process estimators of Gaussian optical field intensities (Corresp.)

    Publication Year: 1979 , Page(s): 620 - 624
    Cited by:  Papers (1)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (832 KB)  

    A remarkably simple stochastic equation of evolution for approximately minimum mean square error estimates of the intensity of a Gaussian optical field is obtained, and its performance experimentally demonstrated. The estimator is based on observations of individual photoelectron occurrence times. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.

Aims & Scope

IEEE Transactions on Information Theory publishes papers concerned with the transmission, processing, and utilization of information.

Full Aims & Scope

Meet Our Editors

Editor-in-Chief
Frank R. Kschischang

Department of Electrical and Computer Engineering