By Topic

Information Theory, IEEE Transactions on

Issue 3 • Date May 1997

Filter Results

Displaying Results 1 - 25 of 36
  • Correction to "On the existence of positive-definite maximum-likelihood estimates of structured covariance matrices"

    Publication Year: 1997 , Page(s): 1094 - 1096
    Cited by:  Papers (2)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (113 KB)  

    A correction to, and a more complete proof of, Theorem 1 of Fuhrmann and Miller (see ibid., vol.34, no.7, p.722-29, 1988) is given. We consider the existence of solutions to the maximum-likelihood (ML) structured covariance estimation problem. In particular, we are interested in determining the circumstances under which the log likelihood is unbounded above for a given covariance constraint set R and a data set x/sub 1/...x/sub M/. Theorem 1 states that, provided certain conditions are met by the constraint set R, the log likelihood is unbounded above if and only if there exists some singular matrix R/sub 0/ in R whose range space contains the data. We refer to the latter occurrence as the failure condition of the ML covariance estimation problem. This result has a certain intuitive appeal; however, one must take care in specifying the conditions on R for this to be precisely correct. In the statement of Theorem 1, two restrictions are given, one for sufficiency and one for necessity. Both restrictions as stated are in error. In this article we propose modified restrictions on the space of covariances R which will correct these errors. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Contributors

    Publication Year: 1997 , Page(s): 1097 - 1100
    Save to Project icon | Request Permissions | PDF file iconPDF (37 KB)  
    Freely Available from IEEE
  • Rate-distortion theory for the Shannon cipher system

    Publication Year: 1997 , Page(s): 827 - 835
    Cited by:  Papers (45)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (600 KB)  

    Rate distortion theory is considered for the Shannon cipher system (SCS). The admissible region of cryptogram rate R, key rate Rk, legitimate receiver's distortion D, and wiretapper's uncertainty h is determined for the SCS with a noisy channel. Furthermore, inner and outer bounds of the admissible region of R, Rk, D, and wiretapper's attainable minimum distortion D˜ are derived for the SCS with a finite discrete source and a noiseless channel View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • The empirical distribution of good codes

    Publication Year: 1997 , Page(s): 836 - 846
    Cited by:  Papers (43)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (472 KB)  

    Let the kth-order empirical distribution of a code be defined as the proportion of k-strings anywhere in the codebook equal to every given k-string. We show that for any fixed k, the kth-order empirical distribution of any good code (i.e., a code approaching capacity with vanishing probability of error) converges in the sense of divergence to the set of input distributions that maximize the input/output mutual information of k channel uses. This statement is proved for discrete memoryless channels as well as a large class of channels with memory. If k grows logarithmically (or faster) with blocklength, the result no longer holds for certain good codes, whereas for other good codes, the result can be shown for k growing as fast as a certain fraction of blocklength View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • The Kolmogorov complexity, universal distribution, and coding theorem for generalized length functions

    Publication Year: 1997 , Page(s): 816 - 826
    Cited by:  Papers (1)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (740 KB)  

    A function h(w) is said to be useful for the coding theorem if the coding theorem remains to be true when the lengths |w| of codewords w in it are replaced with h(w). For a codeword w=a0a1...am-1 of length m and an infinite sequence Q=(q0, q1, q2, ...) of real numbers such that 0<qn⩽½, let |w|Q denote the value Σn=0m-1 (if an =0 then -log2qn, else -log2(1-q n)), that is, -log2, (the probability that flippings of coins generate x) assuming that the (i+1)th coin generates a tail (or 0) with probability qi. It is known that if 0<lim infn→∞ qn then |w|Q is useful for the coding theorem and if limn→∞ q n/(1/(2n))=0 then |w|Q is not useful. We introduce several variations of the coding theorem and show sufficient conditions for h(w) not to be useful for these variations. The usefulness is also defined for the expressions that define the Kolmogorov complexity and the universal distribution. We consider the relation among the usefulness for the coding theorem, that for the Kolmogorov complexity, and that for the universal distribution View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A construction of codes with exponential error bounds on arbitrary discrete memoryless channels

    Publication Year: 1997 , Page(s): 992 - 996
    Cited by:  Papers (1)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (236 KB)  

    This correspondence proposes an explicit construction of codes achieving capacity for arbitrary discrete memoryless channels. The proposed code is obtained by concatenating variable inner codes and an algebraic geometry code. Further, we clarify that the proposed code achieves the error exponent obtained by Forney for concatenated codes View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Improved bounds on maximum size binary radar arrays

    Publication Year: 1997 , Page(s): 997 - 1000
    Cited by:  Papers (2)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (184 KB)  

    The maximum size of binary radar arrays (matrices) with only eight or fewer rows has previously been determined. We determine the maximum size of radar arrays containing 9-16 rows, and for those containing 17 rows we narrow the maximum size down to two values. We also give improved upper and lower asymptotic bounds on the maximum size of radar arrays, which narrow the gap between the existing upper and lower asymptotic bounds by more than 25% View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • The role of the asymptotic equipartition property in noiseless source coding

    Publication Year: 1997 , Page(s): 847 - 857
    Cited by:  Papers (11)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (448 KB)  

    The (noiseless) fixed-length source coding theorem states that, except for outcomes in a set of vanishing probability, a source can be encoded at its entropy but not more efficiently. It is well known that the asymptotic equipartition property (AEP) is a sufficient condition for a source to be encodable at its entropy. This paper shows that the AEP is necessary for the source coding theorem to hold for nonzero-entropy finite-alphabet sources. Furthermore, we show that a nonzero-entropy finite-alphabet source satisfies the direct coding theorem if and only if it satisfies the strong converse. In addition, we introduce the more general setting of nonserial information sources which need not put out strings of symbols. In this context, which encompasses the conventional serial setting, the AEP is equivalent to the validity of the strong coding theorem. Fundamental limits for data compression of nonserial information sources are shown based on the flat-top property-a new sufficient condition for the AEP View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Aperiodic autocorrelation and crosscorrelation of polyphase sequences

    Publication Year: 1997 , Page(s): 1000 - 1007
    Cited by:  Papers (10)  |  Patents (1)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (332 KB)  

    A detailed analysis of the maximum aperiodic autocorrelation of the original Chu sequences (equivalently, P3/P4 pulse compression codes) is presented. The result implies the best known upper bound on the minimax aperiodic autocorrelation for polyphase sequences except when the length is very small or a perfect square. It is well known that determining the minimax aperiodic correlation for polyphase sequence sets is an intractable task. The simplest nontrivial cases for Barker and general polyphase sequences are solved for the first time View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Improved bounds for error recovery times of decision feedback equalization

    Publication Year: 1997 , Page(s): 890 - 902
    Cited by:  Papers (4)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (452 KB)  

    New upper and lower bounds on the mean recovery time of decision feedback equalization (DFE) are derived. The recovery time is defined as the time it takes the decision feedback equalizer (DFEQ) to reach the error-free state after an error has corrupted an error-free DFEQ. The derivations of the bounds assume a causal channel response, independent data symbols, and independent noise samples. The bounds are found to be tighter, especially at large SNR, than previous bounds in a numerical example View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Estimates for the range of binomiality in codes' spectra

    Publication Year: 1997 , Page(s): 987 - 991
    Cited by:  Papers (7)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (204 KB)  

    We derive new estimates for the range of binomiality in a code's spectra, where the distance distribution of a code is upperbounded by the corresponding normalized binomial distribution. The estimates depend on the code's dual distance View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Decision region approximation by polynomials or neural networks

    Publication Year: 1997 , Page(s): 903 - 907
    Cited by:  Papers (2)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (176 KB)  

    We give degree of approximation results for decision regions which are defined by polynomial and neural network parametrizations. The volume of the misclassified region is used to measure the approximation error, and results for the degree of L1 approximation of functions are used. For polynomial parametrizations, we show that the degree of approximation is at least 1, whereas for neural network parametrizations we prove the slightly weaker result that the degree of approximation is at least r, where r can be any number in the open interval (0, 1) View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Information and entropy of continuous random variables

    Publication Year: 1997 , Page(s): 1080 - 1084
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (200 KB)  

    The mean value of the square of a generalized score function is shown to be interpretable as information associated with a continuous random variable. This information is in particular cases equal to the Fisher information of the corresponding distribution View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A new class of nonlinear PN sequences over GF(qn)

    Publication Year: 1997 , Page(s): 1007 - 1012
    Cited by:  Papers (2)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (252 KB)  

    A new class of pseudo-random sequences (called cascaded GMW-type permutation polynomial (CGPP) sequences) over GP(qn) is constructed using a system of n orthogonal cascaded GMW functions in GF(q). Their statistical properties are given. Classification, evaluation, and implementation of the CGPP sequences are derived. A sequence generated by mapping elements of a CGPP sequence into GF(q) is also presented View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Quasi-cyclic structure of Reed-Muller codes and their smallest regular trellis diagram

    Publication Year: 1997 , Page(s): 1040 - 1052
    Cited by:  Papers (5)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (536 KB)  

    The largest quasi-cyclic subcode of the Reed-Muller code R(r,m), invariant under the shift T2m-l is determined. This code, denoted QCR(r,m,l), is presented through its module decomposition into cyclic submodules. The smallest regular trellis diagram (SRTD) is defined for block codes. This trellis and its construction algorithm are given for the class of cyclic-form codes. Using the cyclic-form structure of QCR(r,m,l), the 2l-section SRTD of this code is determined. The eight-section SRTD is given for the Reed-Muller codes R(r,m). The quasi-cyclic subcodes of R(r,m) with regular 2l-section minimal trellis diagrams are presented View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Lengthening and the Gilbert-Varshamov bound

    Publication Year: 1997 , Page(s): 991 - 992
    Cited by:  Papers (3)  |  Patents (1)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (72 KB)  

    We use lengthening and an enhanced version of the Gilbert-Varshamov lower bound for linear codes to construct a large number of record-breaking codes. Our main theorem may be seen as a closure operation on databases View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Bounds on the accuracy of estimating the parameters of discrete homogeneous random fields with mixed spectral distributions

    Publication Year: 1997 , Page(s): 908 - 922
    Cited by:  Papers (7)  |  Patents (1)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (776 KB)  

    This paper considers the achievable accuracy in jointly estimating the parameters of a real-valued two-dimensional (2-D) homogeneous random field with mixed spectral distribution, from a single observed realization of it. On the basis of a 2-D Wold-like decomposition, the field is represented as a sum of mutually orthogonal components of three types: purely indeterministic, harmonic, and evanescent. An exact form of the Cramer-Rao lower bound on the error variance in jointly estimating the parameters of the different components is derived. It is shown that the estimation of the harmonic component is decoupled from that of the purely indeterministic and the evanescent components. Moreover, the bound on the parameters of the purely indeterministic and the evanescent components is independent of the harmonic component. Numerical evaluation of the bounds provides some insight into the effects of various parameters on the achievable estimation accuracy View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Symmetrical multilevel diversity coding

    Publication Year: 1997 , Page(s): 1059 - 1064
    Cited by:  Papers (29)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (216 KB)  

    Multilevel diversity coding was introduced in recent work by Roche (1992) and Yeung (1995). In a multilevel diversity coding system, an information source is encoded by a number of encoders. There is a set of decoders, partitioned into multiple levels, with each decoder having access to a certain subset of the encoders. The reconstructions of the source by decoders within the same level are identical and are subject to the same distortion criterion. Inspired by applications in computer communication and fault-tolerant data retrieval, we study a multilevel diversity coding problem with three levels for which the connectivity between the encoders and decoders is symmetrical. We obtain a single-letter characterization of the coding rate region and show that coding by superposition is optimal for this problem. Generalizing to a symmetrical problem with an arbitrary number of levels, we derive a tight lower bound on the coding rate sum View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • New code parameters from Reed-Solomon subfield codes

    Publication Year: 1997 , Page(s): 953 - 968
    Cited by:  Papers (5)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (952 KB)  

    We determine the dimensions of subfield codes of Reed-Solomon codes and construct certain extensions and lengthenings of these codes. We start from the duals, using the language of orthogonal arrays. As a first result this allows us to obtain a fair number of improvements in the list of binary, ternary, and quaternary linear codes with largest known minimal distance View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Stability analysis of quota allocation access protocols in ring networks with spatial reuse

    Publication Year: 1997 , Page(s): 923 - 937
    Cited by:  Papers (2)  |  Patents (1)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (692 KB)  

    We consider a slotted ring that allows simultaneous transmissions of messages by different nodes, known as ring with spatial reuse. To alleviate fairness problems that arise in such networks, policies have been proposed that operate in cycles and guarantee that a certain number of packets, not exceeding a given number called a quota, will be transmitted by every node in every cycle. We provide sufficient and necessary stability conditions that implicitly characterize the stability region for such rings. These conditions are derived by extending a technique developed for some networks of queues satisfying a monotonicity property. Our approach to instability is novel and its peculiar property is that it is derived from the instability of a dominant system. Interestingly, the stability region depends on the entire distribution of the message arrival process and the steady-state average cycle lengths of lower dimensional systems, leading to a region with nonlinear boundaries, the exact computation of which is in general intractable. Next, we introduce the notions of essential and absolute stability region. An arrival rate vector belongs to the former region if the system is stable under any arrival distribution with this arrival vector, while it belongs to the latter if there exists some distribution with this rate vector for which the system is stable. Using a linear programming approach, we derive bounds for these stability regions that depend only on conditional average cycle lengths. For the case of two nodes, we provide closed-form expressions for the essential stability region View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A 2-adic approach to the analysis of cyclic codes

    Publication Year: 1997 , Page(s): 977 - 986
    Cited by:  Papers (12)  |  Patents (1)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (568 KB)  

    This paper describes how 2-adic numbers can be used to analyze the structure of binary cyclic codes and of cyclic codes defined over Z 2(a), a⩾2, the ring of integers modulo 2a. It provides a 2-adic proof of a theorem of McEliece that characterizes the possible Hamming weights that can appear in a binary cyclic code. A generalization of this theorem is derived that applies to cyclic codes over Z2(a) that are obtained from binary cyclic codes by a sequence of Hensel lifts. This generalization characterizes the number of times a residue modulo 2a appears as a component of an arbitrary codeword in the cyclic code. The limit of the sequence of Hensel lifts is a universal code defined over the 2-adic integers. This code was first introduced by Calderbank and Sloane (1995), and is the main subject of this paper. Binary cyclic codes and cyclic codes over Z2(a) are obtained from these universal codes by reduction modulo some power of 2. A special case of particular interest is cyclic codes over Z4 that are obtained from binary cyclic codes by means of a single Hensel lift. The binary images of such codes under the Gray isometry include the Kerdock, Preparata, and Delsart-Goethals codes. These are nonlinear binary codes that contain more codewords than any linear code presently known. Fundamental understanding of the composition of codewords in cyclic codes over Z4 is central to the search for more families of optimal codes. This paper also constructs even unimodular lattices from the Hensel lift of extended binary cyclic codes that are self-dual with all Hamming weights divisible by 4. The Leech lattice arises in this way as do extremal lattices in dimensions 32 through 48 View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Algebraic tools to build modulation schemes for fading channels

    Publication Year: 1997 , Page(s): 938 - 952
    Cited by:  Papers (177)  |  Patents (18)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (760 KB)  

    A unified framework is presented in order to build lattice constellations matched to both the Rayleigh fading channel and the Gaussian channel. The method encompasses the situations where the interleaving is done on the real components or on two-dimensional signals. In the latter case, a simple construction of lattices congruent to the densest binary lattices with respect to the Euclidean distance is proposed. It generalizes, in a sense to be clarified later, the structural construction proposed by Forney (1991). These constellations are next combined with coset codes. The partitioning rules and the gain formula are similar to those used for the Gaussian channel View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Adaptive de-noising of signals satisfying differential inequalities

    Publication Year: 1997 , Page(s): 872 - 889
    Cited by:  Papers (10)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1144 KB)  

    The paper is devoted to spatial adaptive estimation of signals satisfying linear differential inequalities with an unknown differential operator of a given order. The classes of signals under consideration cover a wide variety of classes common to nonparametric regression. In particular, they contain the signals whose parameters of smoothness are not uniformly bounded, even locally. We develop an estimator which is optimal in order over a wide range of the classes and “discrete” global accuracy measures View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • On the norm and covering radius of the first-order Reed-Muller codes

    Publication Year: 1997 , Page(s): 1025 - 1027
    Cited by:  Papers (3)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (148 KB)  

    Let ρ(1,m) and N(1,m) be the covering radius and norm of the first-order Reed-Muller code R(1,m), respectively. It is known that ρ(1,2k+1)⩽lower bound [22k-2(2k-1/2)] and N(1,2k+1)⩽2 lower bound [22k-2(2k-1/2)] (k>0). We prove that ρ(1,2k+1)⩽2 lower bound [22k-1-2(2k-3/2)] and N(1,2k+1)⩽4 lower bound [22k-1-2(2k-3/2)] (k>0). We also discuss the connections of the two new bounds with other coding theoretic problems View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Probability of error in MMSE multiuser detection

    Publication Year: 1997 , Page(s): 858 - 871
    Cited by:  Papers (441)  |  Patents (17)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (596 KB)  

    The performance analysis of the minimum-mean-square-error (MMSE) linear multiuser detector is considered in an environment of nonorthogonal signaling and additive white Gaussian noise. In particular, the behavior of the multiple-access interference (MAI) at the output of the MMSE detector is examined under various asymptotic conditions, including: large signal-to-noise ratio; large near-far ratios; and large numbers of users. These results suggest that the MAI-plus-noise contending with the demodulation of a desired user is approximately Gaussian in many cases of interest. For the particular case of two users, it is shown that the maximum divergence between the output MAI-plus-noise and a Gaussian distribution having the same mean and variance is quite small in most cases of interest. It is further proved in this two-user case that the probability of error of the MMSE detector is better than that of the decorrelating linear detector for all values of normalized crosscorrelations not greater than ½√(2+√3)≅0.9659 View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.

Aims & Scope

IEEE Transactions on Information Theory publishes papers concerned with the transmission, processing, and utilization of information.

Full Aims & Scope

Meet Our Editors

Editor-in-Chief
Frank R. Kschischang

Department of Electrical and Computer Engineering