By Topic

Information Theory, IEEE Transactions on

Issue 6 • Date Nov 1990

Filter Results

Displaying Results 1 - 25 of 34
  • An algorithm for near-optimal placement of sensor elements

    Page(s): 1280 - 1284
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (400 KB)  

    An important question in array design is that of where to place the elements of a sparse array for optimal performance in terms of its ability to detect and resolve a greater number of sources than conventionally possible. In particular, it has been shown that when sensor elements are arranged in the minimum redundancy fashion, by performing an augmentation technique on the covariances obtained from the array outputs, an M element array can be made to estimate the directions of arrival of as many as M(M-1)/2 uncorrelated sources unambiguously. Constructive procedures are developed to evaluate integer locations for an array of given sensors that span a prescribed distance, such that any missing integer is expressible as the difference of two sensor locations. New upper bounds for the ratio of the square of the minimum number of elements to the spanning distance are also established View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Asymptotic bounds on the covering radius of binary codes

    Page(s): 1470 - 1472
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (236 KB)  

    Asymptotic covering properties of families of binary codes are studied. A Gaussian approximation to the weight distribution of translates for codes of high strength is used. From this an upper bound on the covering radius of these codes is deduced. Applications include Reed-Muller codes, quadratic residue codes, and BCH codes. Sufficient conditions for a family of codes to have best possible covering radius (asymptotically perfect codes) are derived View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • An upper bound on the covering radius as a function of the dual distance

    Page(s): 1472 - 1474
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (228 KB)  

    P. Delsarte (1973) developed a method that gives an upper bound on the cardinality of a code as a function of its minimum distance. It is shown that, using a modification of that method, one gets an upper bound on the covering radius of a code as a function of its dual distance. As an interesting special case, the covering radius of the dual of a BCH code is considered View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A new table of constant weight codes

    Page(s): 1334 - 1380
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (3684 KB)  

    A table of binary constant weight codes of length n⩽28 is presented. Explicit constructions are given for most of the 600 codes in the table; the majority of these codes are new. The known techniques for constructing constant weight codes are surveyed, and a table of (unrestricted) binary codes of length n⩽28 is given View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Any code of which we cannot think is good

    Page(s): 1453 - 1461
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (860 KB)  

    A central paradox of coding theory concerns the existence and construction of the best codes. Virtually every linear code is good, in the sense that it meets the Gilbert-Varshamov bound on distance versus redundancy. Despite the sophisticated constructions for codes derived over the years, however, no one has succeeded in demonstrating a constructive procedure that yields such codes over arbitrary symbol fields. Using the theory of Kolmogorov complexity, it is shown that this statement holds true in a rigorous mathematical sense: any linear code that is truly random, in the sense that there is no concise way of specifying the code, is good. Furthermore, random selection of a code that does contain some constructive pattern results, with probability bounded away from zero, in a code that does not meet the Gilbert-Varshamov bound regardless of the block length of the code. In contrast to the situation for linear codes, it is shown that there are effectively random nonlinear codes which have no guarantee on distance. In addition, it is shown that the techniques of Kolmogorov complexity can be used to derive typical properties of classes of codes in a novel way View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • On the capacity of the Gaussian channel with a finite number of input levels

    Page(s): 1426 - 1428
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (192 KB)  

    An analytic confirmation of an observation made by G. Ungerboeck (1965) that approximately optimal performance on a Gaussian channel with capacity C can be obtained with about 2C+1 levels is given. In particular, the results imply that by using about 2C-1 levels, a rate of nearly C-1 bit can be achieved, and that by using 2C+1 levels a rate of about C-0.4 bits can be achieved View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • On a class of multilevel universal source coding schemes

    Page(s): 1442 - 1446
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (380 KB)  

    This work seeks to reduce the computational complexity and the coding rate associated with a class of universal source coding schemes for encoding fixed length m-ary sequences into variable length sequences. The Lynch-Davisson (1966) minimax universal source coding scheme, the Tanaka-Ando-Leon-Garcia (TAL) weighted universal source coding scheme (1985), and Cover's general enumerative source coding scheme (1973) are modified. The essence of the proposed modification is to rename the m symbols of the alphabet so that the most frequent symbol of the given input data block will be denoted 0 the second most frequent symbol will be denoted 1, and so on, until the least frequent symbol will be denoted m-1. It is proven that out of all the m permutations of the m symbols, the proposed one minimizes the computational complexity of the above mentioned schemes, and preserves their unique decodability property. Moreover, for the modified TAL scheme it is proven that for block lengths nm further data compression can be achieved View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Constructions and bounds for systematic tEC/AUED codes

    Page(s): 1381 - 1390
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (712 KB)  

    Several methods of constructing systematic t-error correcting/all unidirectional error-detecting codes are described. These codes can be constructed by adding a tail to a linear t-error correcting code, but other constructions presented are more of an ad hoc nature. These codes will often be found as suitably chosen subsets of nonsystematic tEC/AUED codes. Further bounds on the word length of systematic tEC/AUED codes are derived, and extensive tables are given View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A lower bound on average codeword length of variable length error-correcting codes

    Page(s): 1474 - 1475
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (156 KB)  

    Codes of variable length that are capable of error correction are studied. A lower bound on the average length of such codes under the criterion of promptness is obtained. This generalizes the celebrated result due to C.E. Shannon (1948) for the noiseless case. The bound obtained provides a measure of optimality for variable-length, error-correcting codes View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Convergence of the projection method for an autoregressive process and a matched DPCM code

    Page(s): 1255 - 1264
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (692 KB)  

    The key step in the analysis of differential pulse code modulation (DPCM) is to find the steady-state probability distribution of a random process in terms of which the code distortion can be evaluated. For the case of an autoregressive source and a matched DPCM code, a well-known approximation technique has been used for the evaluation of the steady-state distribution of the prediction error process. However, the validity of this approximation method has not been justified before. A framework in which this approximation technique can be viewed as the projection method for the solution of integral equations is established. Sufficient conditions under which the approximation method can be rigorously justified are obtained View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Bit probabilities of optimal binary source codes

    Page(s): 1446 - 1450
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (484 KB)  

    The probabilities of the bits produced by an optimal binary source encoder for a memoryless source are analyzed. The class of sources for which the probability of zero must equal 1/2 for any such encoder is described. Procedures that minimize (maximize) the absolute difference of the bit probabilities are discussed. Corresponding upper bounds are derived and compared for the class of uniform sources View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Performance of global load balancing by local adjustment

    Page(s): 1398 - 1414
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1316 KB)  

    A set of M resource locations and a set of αM consumers are given. Each consumer requires a specified amount of resource, and is constrained to obtain the resource from a specified subset of locations. The problem of assigning consumers to resource locations so as to balance the load among the resource locations as much as possible is considered. It is shown that there are assignments, termed uniformly most-balanced assignments, that simultaneously minimize certain symmetric, separable, convex cost functions. The problem of finding such assignments is equivalent to a network flow problem with convex cost. Algorithms of both the iterative and combinatorial type are given for computing the assignments. The distribution function of the load at a given location for a uniformly most-balanced assignment is studied, assuming that the set of locations each consumer can use is random. An asymptotic lower bound on the distribution function is given for M tending to infinity, and an upper bound is given on the probable maximum load. It is shown that there is typically a large set of resource locations that all have the minimum load, and that for large average loads the maximum load is near the average load View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A new upper bound on the minimal distance of self-dual codes

    Page(s): 1319 - 1333
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1076 KB)  

    It is shown that the minimal distance d of a binary self-dual code of length n⩾74 is at most 2[(n+6)/10]. This bound is a consequence of some new conditions on the weight enumerator of a self-dual code obtained by considering a particular translate of the code, called its shadow. These conditions also enable one to find the highest possible minimal distance of a self-dual code for all n⩾60; to show that self-dual codes with d⩽6 exist precisely for n⩾22, with d ⩾8 exist precisely for n=24, 32 and n⩾26, and with d⩾10 exist precisely for n⩾46; and to show that there are exactly eight self-dual codes of length 32 with d=8. Several of the self-dual codes of length 34 have trivial group (this appears to be the smallest length where this can happen) View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Analysis of a modified Hebbian rule

    Page(s): 1391 - 1397
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (504 KB)  

    A modified Hebbian rule using the matrix G=sgn(X TX) to induce a certain mapping is discussed. This mapping g is specified as soon as one has chosen the m×n matrix X over U to construct G by use of the above expression. The analysis of g relies on simple counting arguments and on the use of Stirling's approximation to obtain asymptotic results View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Finding a basis for the characteristic ideal of an n-dimensional linear recurring sequence

    Page(s): 1480 - 1487
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (680 KB)  

    An n-dimensional linear recurring sequence (σ) of elements from a field F, for n⩾1 is considered. An algorithm χ-BASE which determines a basis for the ideal characteristic polynomials of (σ) under certain reasonable conditions is presented. This analysis applied to doubly periodic arrays in particular View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Optimum `1'-ended binary prefix codes

    Page(s): 1435 - 1441
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (460 KB)  

    The problem of finding a binary prefix code of minimum average codeword length for a given finite probability distribution subject to the requirement that each codeword must end with a 1 is considered. Lower and upper bounds to the performance of the optimum code are derived; the lower bound is tight for certain probability distributions. An algorithm that generates an optimum code for any given distribution is described View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • An improved sphere covering bound for the codes with n=3 R+2

    Page(s): 1476 - 1478
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (188 KB)  

    Let C be a binary code (not necessarily linear) with covering radius R and length n=3R+2. The sphere covering bound on the cardinality of C is improved considerably provided C has minimal distance d>2. Some new results on the function t[n,k] (the smallest covering radius of any binary linear code with length n and dimension k): t[38.6]⩾13, t[47.7]⩾16, t[59.8]⩾20 are given View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • On the unnormalized solution of the filtering problem with counting process observations

    Page(s): 1415 - 1425
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (748 KB)  

    A general procedure to solve filtering problems for counting process observations is discussed. Linear (nonstochastic, integro-differential) equations describe the evolution of unnormalized conditional distribution of the state process between observation jump times, while at jump times a linear updating is required. Final normalization is the only nonlinear operation to be implemented. Quite general situations may be accommodated in the present setup; the state can be virtually any Markov semimartingale, the observation process may affect the dynamics of the state and vice versa, and there is complete freedom in correlating state and observation martingale terms View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A class of codes for a T active users out of N multiple-access communication system

    Page(s): 1206 - 1219
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (868 KB)  

    A code construction for a T active users out of N multiple-access system (TANMAS) is discussed. The multiple-access channel (MAC) that is used in the TANMAS is a discrete-time noiseless real adder channel used without feedback with N real or binary inputs. Each input may be affected by an unknown, slowly varying channel gain and by an unknown, slowly varying channel offset. A set of N codes is constructed such that the sum of codewords of any known set of T or less active users is uniquely decodable. The sum rate of the codes approaches one from above if the input alphabet of the MAC is binary, and it approaches one from below if the input symbols are real numbers. For the case when the set of active users is unknown. For the case when the set of active users is unknown, it is shown that this set can be identified uniquely (in addition to unique decodability) provided that at most T/2 users are active simultaneously. The sum rate is then reduced to 1/2, which is approached from above and from below, respectively, for binary and for real channel inputs. A simple decoding algorithm which operates over the reals is given for the case when the set of active users is known. It is pointed out that one of the applications of the given codes is in hybrid multiple-access systems that use both multiple-access coding and collision resolution View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • On optimal quantization of noisy sources

    Page(s): 1450 - 1452
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (232 KB)  

    Problems in optimal multidimensional quantization of sources corrupted by noise are addressed. Expressions for the optimum quantizer values and the optimum quantization rule for the weighted squared error distortion measure are found and calculated for the Gaussian signal in additive independent Gaussian noise problem. Some properties of the optimum quantizer, and its relations with the optimal estimator for the general problem, are derived View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • One universally efficient estimation of the first-order autoregressive parameter and universal data compression

    Page(s): 1245 - 1254
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (620 KB)  

    A universal nearly efficient estimator is proposed for the first-order autoregressive (AR) model where the probability distribution of the driving noise is unknown. It is shown that universal estimators for the AR model can be derived from universal data compression algorithms and universal tests for randomness. In other words, estimators derived appropriately from efficient universal codes can be expected to inherit good estimation performance under some conditions. The proposed estimator has a simple information-theoretic interpretation related to universal coding, which can be easily generalized to the higher-order case and to other parametric models, e.g. the one-sample location model, the two-sample location model, and the linear regression model View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • The trace description of irreducible quasi-cyclic codes

    Page(s): 1463 - 1466
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (348 KB)  

    The notion of a q-ary irreducible quasi-cyclic code of block length n and index r is introduced. A trace description of such a code is provided in a fashion similar to the trace description of irreducible cyclic codes. In particular, it is shown that an irreducible quasi-cyclic code of dimension k is completely described by an irreducible cyclic code and r elements from a field of cardinality qk. Using this fact, a number of binary irreducible quasi-cyclic codes of index 2 are constructed and their weight spectra obtained View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A new recursive filter for systems with multiplicative noise

    Page(s): 1430 - 1435
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (456 KB)  

    An optimal linear recursive minimum mean-square-error estimator was previously developed by the authors (see IEEE Trans. Autom. Control, vol.34, no.5, p.568-74, May 1989) for a zero-mean signal corrupted by multiplicative noise in its measurement model. This recursive filter cannot be obtained by the recursive structure of a conventional Kalman filter where the new estimate is a linear combination of the previous estimate and the new data. Instead, the recursive structure was achieved by combining the previous estimate with recursive innovation, a linear combination of the most recent two data samples and the previous estimate. In this work the signal is extended to be nonzero-mean. In the conventional Kalman filter, the superposition principle can be applied to both the signal and the measurement models for this nonzero-mean extension. However, when multiplicative noise exists, the measurement model becomes nonlinear. Therefore, a new recursive structure for the innovation process needs to be developed to achieve a recursive filter View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A note on square roots in finite fields

    Page(s): 1494 - 1498
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (444 KB)  

    A simple method showing the quadratic character in a finite field of odd order q can be computed in O(log2 q) steps is presented. It is also shown how sequences generated deterministically from a random seed can be used reliably in a recent randomized algorithm of R. Peralta (1986) for computing square roots in finite fields View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A binary analog to the entropy-power inequality

    Page(s): 1428 - 1430
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (232 KB)  

    Let {Xn}, {Yn} be independent stationary binary random sequences with entropy H( X), H(Y), respectively. Let h(ζ)=-ζlogζ-(1-ζ)log(1-ζ), 0⩽ζ⩽1/2, be the binary entropy function and let σ(X)=h-1 (H(X)), σ(Y)=h-1 (H(Y)). Let zn=XnYn , where ⊕ denotes modulo-2 addition. The following analog of the entropy-power inequality provides a lower bound on H(Z ), the entropy of {Zn}: σ(Z)⩾σ(X)*σ(Y), where σ(Z)=h-1 (H(Z)), and α*β=α(1-β)+β(1-α). When {Y n} are independent identically distributed, this reduces to Mrs. Gerber's Lemma from A.D. Wyner and J. Ziv (1973) View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.

Aims & Scope

IEEE Transactions on Information Theory publishes papers concerned with the transmission, processing, and utilization of information.

Full Aims & Scope

Meet Our Editors

Editor-in-Chief
Frank R. Kschischang

Department of Electrical and Computer Engineering