By Topic

Information Theory, IEEE Transactions on

Issue 1 • Date Jan. 1998

Filter Results

Displaying Results 1 - 25 of 47
  • Full text access may be available. Click article title to sign in or learn about subscription options.
  • Maximum disjoint bases and constant-weight codes

    Page(s): 333 - 334
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (88 KB)  

    The following lower bound for binary constant weight codes are derived by an explicit construction: A(17,4,5)⩾441. The construction exploits maximal sets of bases in the four-dimensional binary vector space pairwise intersecting in at most two vectors View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Quantized overcomplete expansions in IRN: analysis, synthesis, and algorithms

    Page(s): 16 - 31
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (652 KB)  

    Coefficient quantization has peculiar qualitative effects on representations of vectors in IR with respect to overcomplete sets of vectors. These effects are investigated in two settings: frame expansions (representations obtained by forming inner products with each element of the set) and matching pursuit expansions (approximations obtained by greedily forming linear combinations). In both cases, based on the concept of consistency, it is shown that traditional linear reconstruction methods are suboptimal, and better consistent reconstruction algorithms are given. The proposed consistent reconstruction algorithms were in each case implemented, and experimental results are included. For frame expansions, results are proven to bound distortion as a function of frame redundancy r and quantization step size for linear, consistent, and optimal reconstruction methods. Taken together, these suggest that optimal reconstruction methods will yield O(1/r2) mean-squared error (MSE), and that consistency is sufficient to insure this asymptotic behavior. A result on the asymptotic tightness of random frames is also proven. Applicability of quantized matching pursuit to lossy vector compression is explored. Experiments demonstrate the likelihood that a linear reconstruction is inconsistent, the MSE reduction obtained with a nonlinear (consistent) reconstruction algorithm, and generally competitive performance at low bit rates View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A link between quasi-cyclic codes and convolutional codes

    Page(s): 431 - 435
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (156 KB)  

    Given a quasi-cyclic code with minimum Hamming distance d, a set of convolutional codes is derived with free distance equal to d. It is shown that an increase in the rate of these codes results in a decrease in the memory length. The connection between these codes is illustrated with several well-known quasi-cyclic codes. The free distance of some partial unit memory convolutional codes can be determined using the results in this correspondence View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Estimation of 1/f noise

    Page(s): 32 - 46
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (756 KB)  

    Several models have emerged for describing 1/fγ noise processes. Based on these, various techniques for estimating the properties of such processes have been developed. This paper provides theoretical analysis of a new wavelet-based approach which has the advantages of having low computational complexity and being able to handle the case where the 1/fγ noise might be embedded in a further white-noise process. However, the analysis conducted here shows that these advantages are balanced by the fact that the wavelet-based scheme is only consistent for spectral exponents γ in the range γ∈(0, 1). This is in contradiction to the results suggested in previous empirical studies. When γ∈(0, 1) this paper also establishes that wavelet-based maximum-likelihood methods are asymptotically Gaussian and efficient. Finally, the asymptotic rate of mean-square convergence of the parameter estimates is established and is shown to slow as γ approaches one. Combined with a survey of non-wavelet-based methods, these new results give a perspective on the various tradeoffs to be considered when modeling and estimating 1/fγ noise processes View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Common randomness in information theory and cryptography. II. CR capacity

    Page(s): 225 - 240
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (924 KB)  

    For pt.I see ibid., vol.39, p.1121, 1993. The common randomness (CR) capacity of a two-terminal model is defined as the maximum rate of common randomness that the terminals can generate using resources specified by the given model. We determine CR capacity for several models, including those whose statistics depend on unknown parameters. The CR capacity is shown to be achievable robustly, by common randomness of nearly uniform distribution no matter what the unknown parameters are. Our CR capacity results are relevant for the problem of identification capacity, and also yield a new result on the regular (transmission) capacity of arbitrarily varying channels with feedback View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Laplace's law of succession and universal encoding

    Page(s): 296 - 303
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (332 KB)  

    This correspondence shows that the known “add-half” rule is not asymptotically optimal for predicting the (n+1)st symbol after a sequence of n symbols, whereas the “add-β0” rule, β0=0.50922···is View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Asymptotic analysis of multiple description quantizers

    Page(s): 278 - 284
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (288 KB)  

    A high-rate analysis of multiple description quantizers is presented for rth-power distortions and general source densities. Both, fixed-length and variable-length encoding of the quantizer indices are considered. Optimal companding functions are shown to be the same as for single-channel quantizers. As compared to the bound of Ozarow (1980), a gap of 8.69 dB and 3.07 dB exists between the entropy-constrained and level-constrained cases, respectively, for a memoryless Gaussian source and r=2 View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A new algorithm for finding minimum-weight words in a linear code: application to McEliece's cryptosystem and to narrow-sense BCH codes of length 511

    Page(s): 367 - 378
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (436 KB)  

    An algorithm for finding minimum-weight words in large linear codes is developed. It improves all previous attacks on the public-key cryptosystems based on codes and it notably points out some weaknesses in McEliece's (1978) cipher. We also determine with it the minimum distance of some BCH codes of length 511 View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Generalized Hamming weights of q-ary Reed-Muller codes

    Page(s): 181 - 196
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (988 KB)  

    The order bound on generalized Hamming weights is introduced in a general setting of codes on varieties which comprises both the one point geometric Goppa codes as well as the q-ary Reed-Muller codes. For the latter codes it is shown that this bound is sharp and that they satisfy the double chain condition View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Binary lattice vector quantization with linear block codes and affine index assignments

    Page(s): 79 - 94
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (732 KB)  

    We determine analytic expressions for the performance of some low-complexity combined source-channel coding systems. The main tool used is the Hadamard transform. In particular, we obtain formulas for the average distortion of binary lattice vector quantization with affine index assignments, linear block channel coding, and a binary-symmetric channel. The distortion formulas are specialized to nonredundant channel codes for a binary-symmetric channel, and then extended to affine index assignments on a binary-asymmetric channel. Various structured index assignments are compared. Our analytic formulas provide a computationally efficient method for determining the performance of various coding schemes. One interesting result shown is that for a uniform source and uniform quantizer, the natural binary code is never optimal for a nonsymmetric channel, even though it is known to be optimal for a symmetric channel View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Capacity of multilevel threshold devices

    Page(s): 241 - 255
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (736 KB)  

    The principal purpose of this research is to discover the underlying properties of linear multilevel threshold devices and networks of linear multilevel threshold devices. The main theoretical developments of this investigation entail generalizing the function-counting theorem associated with linear bi-level threshold devices to that of linear multilevel threshold devices. The investigation reveals surprising connections that underlie linear inequalities, linear separability, linear ordering, and region counting. The results have implications in the field of geometric probability. Computations based on theoretical analysis support Brown's (1964) conjecture and suggest that k/(k-1) is a natural definition for the information-storage capacity of a k-level threshold device. A lower bound on the number of weights required to implement a universal network of k-level threshold devices is derived. Finally, it is shown that the Vapnik-Chervonenki (1971)s dimension (VC-dimension) for the class of multilevel threshold function is d+1 for pattern vectors in ℛd . This VC-dimension is also linked to an error-rate bound for multilevel threshold functions within the framework of uniform learnability View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • On locally invertible rate-1/n convolutional encoders

    Page(s): 420 - 422
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (100 KB)  

    A locally invertible convolutional encoder has a local inverse defined as a full rank w×w matrix that specifies a one-to-one mapping between equal-length blocks of information and encoded bits. In this correspondence, it is shown that a rate-1/n convolutional encoder is nondegenerate and noncatastrophic if and only if it is locally invertible. Local invertibility is used to obtain upper and lower bounds on the number of consecutive zero-weight branches in a convolutional codeword. Further, existence of a local inverse can be used as an alternate test for noncatastrophicity instead of the usual approach involving computation of the greatest common divisor of n polynomials View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Codes correcting phased burst erasures

    Page(s): 416 - 420
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (236 KB)  

    We introduce a family of binary array codes of size t×n, correcting multiple phased burst erasures of size t. The codes achieve maximal correcting capability, i.e., being considered as codes over GF(2 t) they are MDS. The length of the codes is n=Σl=1 L(lt) where L is a constant or is slowly growing in t. The complexity of encoding and decoding is proportional to rnmL where r is the number of correctable erasures, and m is the smallest number such that 2t=1 modulo m. This compares favorably with the complexity of decoding codes obtained from the shortened Reed-Solomon codes having the same parameters View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • On sliding-window universal data compression with limited memory

    Page(s): 66 - 78
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (876 KB)  

    Nonasymptotic coding and converse theorems are derived for universal data compression algorithms in cases where the training sequence (“history”) that is available to the encoder consists of the most recent segment of the input data string that has been processed, but is not large enough so as to yield the ultimate compression, namely, the entropy of the source View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Goppa codes and trace operator

    Page(s): 290 - 294
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (276 KB)  

    We study Goppa codes, Γ(L,g), defined by the polynomial g(z)=a(z)TrFpms:Fps(b(z)). It is shown that the dimension of these codes never reaches the general, well-known, bound for Goppa codes. New bounds are proposed depending on the value of m and p. Furthermore, we prove that when p=2 these codes have only even weights View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Extremal self-dual codes with an automorphism of order 2

    Page(s): 323 - 328
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (352 KB)  

    A method to design binary self-dual codes with an automorphism of order two without fixed points is presented. Extremal self-dual codes with lengths 40, 42, 44, 54, 58, 68 are constructed. Many of them have weight enumerators for which extremal codes were previously not known to exist View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • The common randomness capacity of a pair of independent discrete memoryless channels

    Page(s): 215 - 224
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (632 KB)  

    We study the following problem: two agents Alice and Bob are connected to each other by independent discrete memoryless channels. They wish to generate common randomness, i.e. agree on a common random variable, by communicating interactively over the two channels. Assuming that Alice and Bob are allowed access to independent external random sources at rates (in bits per step of communication) of HA and HB, respectively, we show that they can generate common randomness at a rate of max{min[HA+H(W|Q),I(P;V)]+min[HB +H(V|P), I(Q;W)]} bits per step, by exploiting the noise on the two channels. Here, V is the channel from Alice to Bob, and W is the channel from Bob to Alice. The maximum is over all probability distributions P and Q on the input alphabets of V and W, respectively. We also prove a strong converse which establishes the above rate as the highest attainable in this situation View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • On the weight hierarchy of Goethals codes over Z4

    Page(s): 304 - 307
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (212 KB)  

    The rth generalized Hamming weight dr(m,j) of the Goethals code 𝒢m(j) of length 2m over Z4 is considered in this correspondence. In the case that m⩾3 is an odd integer, dr(m,j) is exactly determined for r=0.5, 1, 1.5, 2, 2.5, and 3.0. For a composite m, we give an upper bound dr (m,j) using the lifting technique View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Identification plus transmission over channels with perfect feedback

    Page(s): 284 - 290
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (372 KB)  

    We determine the region of all identification and transmission rate-pairs achievable over a discrete memoryless channel (DMC) with perfect and instantaneous feedback, for both randomized and deterministic encoding. As a by-product, we also have a new proof of Kemperman's (1973) strong converse to Shannon's coding theorem for DMC's with feedback View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A new algorithm for Golomb ruler derivation and proof of the 19 mark ruler

    Page(s): 379 - 382
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (88 KB)  

    A new parallel distributed algorithm for Golomb (1977) ruler derivation is presented. This algorithm was used to prove computationally the optimality of three rulers. Two of these were previously proven but yet unpublished, and the authors' independent derivation confirmed these results. The last ruler, of 19 marks and size 246, was known to be near-optimal and was computationally proven optimal in this work View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Wavelet analysis of long-range-dependent traffic

    Page(s): 2 - 15
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (452 KB)  

    A wavelet-based tool for the analysis of long-range dependence and a related semi-parametric estimator of the Hurst parameter is introduced. The estimator is shown to be unbiased under very general conditions, and efficient under Gaussian assumptions. It can be implemented very efficiently allowing the direct analysis of very large data sets, and is highly robust against the presence of deterministic trends, as well as allowing their detection and identification. Statistical, computational, and numerical comparisons are made against traditional estimators including that of Whittle. The estimator is used to perform a thorough analysis of the long-range dependence in Ethernet traffic traces. New features are found with important implications for the choice of valid models for performance evaluation. A study of mono versus multifractality is also performed, and a preliminary study of the stationarity with respect to the Hurst parameter and deterministic trends View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • On the complexity of decoding lattices using the Korkin-Zolotarev reduced basis

    Page(s): 162 - 171
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (420 KB)  

    Upper and lower bounds are derived for the decoding complexity of a general lattice L. The bounds are in terms of the dimension n and the coding gain γ of L, and are obtained based on a decoding algorithm which is an improved version of Kannan's (1983) method. The latter is currently the fastest known method for the decoding of a general lattice. For the decoding of a point x, the proposed algorithm recursively searches inside an, n-dimensional rectangular parallelepiped (cube), centered at x, with its edges along the Gram-Schmidt vectors of a proper basis of L. We call algorithms of this type recursive cube search (RCS) algorithms. It is shown that Kannan's algorithm also belongs to this category. The complexity of RCS algorithms is measured in terms of the number of lattice points that need to be examined before a decision is made. To tighten the upper bound on the complexity, we select a lattice basis which is reduced in the sense of Korkin-Zolotarev (1873). It is shown that for any selected basis, the decoding complexity (using RCS algorithms) of any sequence of lattices with possible application in communications (γ⩾1) grows at least exponentially with n and γ. It is observed that the densest lattices, and almost all of the lattices used in communications, e.g., Barnes-Wall lattices and the Leech lattice, have equal successive minima (ESM). For the decoding complexity of ESM lattices, a tighter upper bound and a stronger lower bound result are derived View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Cosets of convolutional codes with least possible maximum zero- and one-run lengths

    Page(s): 423 - 431
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (360 KB)  

    A communication or storage system may use a coset of a binary convolutional code for both symbol synchronization and error control. To facilitate symbol synchronization, the coset must have a short maximum zero-run length Lmax. General upper and lower bounds on Lmax were given previously by Hole. In this correspondence we use these bounds to identify which convolutional codes have cosets with short Lmax. For such a code, we then show how to determine a coset with the least possible Lmax among all cosets of the code. Exact expressions for the least possible Lmax of convolutional code cosets are given, and examples of such cosets with large free distances are tabulated. Bounds on Lmax for cosets of block codes are also provided. It is indicated how to tighten the bounds for block codes satisfying the one-way chain condition. We show that the cosets obtained from traditional high-rate block code constructions have larger Lmax than cosets of convolutional codes with approximately the same rates. In some systems the convolutional code cosets must have short maximum one-run lengths as well as short maximum zero-run lengths to avoid loss of symbol synchronization. It is shown how to determine convolutional codes whose cosets with least possible maximum zero-run lengths also have least possible maximum one-run lengths View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Nonparametric estimation of the cyclic cross spectrum

    Page(s): 351 - 358
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (352 KB)  

    Cyclostationary processes are an important class of nonstationary processes. We consider nonparametric estimation of the cyclic cross spectrum. A smoothed periodogram-based estimator is studied and its asymptotic behavior characterized, extending univariate work to the multivariate case. Application to cyclic coherence measurements is discussed. The results are useful in a variety of multisensor cyclostationary signal processing scenarios such as time delay and bearing estimation View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.

Aims & Scope

IEEE Transactions on Information Theory publishes papers concerned with the transmission, processing, and utilization of information.

Full Aims & Scope

Meet Our Editors

Editor-in-Chief
Frank R. Kschischang

Department of Electrical and Computer Engineering