By Topic

Information Theory, IEEE Transactions on

Issue 4 • Date July 1997

Filter Results

Displaying Results 1 - 25 of 39
  • Contributors

    Page(s): 1382 - 1385
    Save to Project icon | Request Permissions | PDF file iconPDF (32 KB)  
    Freely Available from IEEE
  • A generalized state-splitting algorithm

    Page(s): 1326 - 1338
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (496 KB)  

    We describe a generalization of the state-splitting algorithm (also known as the Adler-Coppersmith-Hassner (1983) algorithm) for constructing encoders which encode arbitrary data into constrained systems of sequences. In the generalized algorithm, we replace approximate eigenvectors with approximate eigenmatrices to yield a framework for designing encoders with smaller sliding-block windows and therefore lower error propagation View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • The depth distribution-a new characterization for linear codes

    Page(s): 1361 - 1363
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (148 KB)  

    We apply the well-known operator of sequences, the derivative D, on codewords of linear codes. The depth of a codeword c is the smallest integer i such that Dic (the derivative applied i consecutive times) is zero. We show that the depth distribution of the nonzero codewords of an [n, k] linear code consists of exactly k nonzero values, and its generator matrix can be constructed from any k nonzero codewords with distinct depths. Interesting properties of some linear codes, and a way to partition equivalent codes into depth-equivalence classes are also discussed View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Arithmetic crosscorrelations of feedback with carry shift register sequences

    Page(s): 1342 - 1345
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (188 KB)  

    An arithmetic version of the crosscorrelation of two sequences is defined, generalizing Mandelbaum's (1967) arithmetic autocorrelations. Large families of sequences are constructed with ideal (vanishing) arithmetic crosscorrelations. These sequences are decimations of the 2-adic expansions of rational numbers p/q such that 2 is a primitive root module q View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • An information-spectrum approach to source coding theorems with a fidelity criterion

    Page(s): 1145 - 1164
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1336 KB)  

    The rate-distortion problem for the general class of nonstationary and/or nonergodic sources with an arbitrary distortion measure (not necessarily additive) is studied. We are especially concerned with the case of variable-rate coding under maximum-distortion criterion. It turns out that, in the framework where we cannot readily invoke the standard asymptotic equipartition property, an information-spectrum approach devised by Han and Verdu (1993) plays the key role in establishing such a general formula. Comparisons with the rate-distortion formulas with fixed-rate coding of Steinberg and Verdu (see ibid., vol.42, no.1, p.63-86, 1996) are also discussed to obtain an insight into the general features of this kind of nonstationary and nonergodic problems View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • The weight distributions of extended binary primitive BCH codes of length 128

    Page(s): 1364 - 1371
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (432 KB)  

    In previous work, a method was presented to compute the weight distribution of a linear block code by using its trellis diagram. In this correspondence, the method is improved by using the trellis structure of linear block codes. Another method with reduced computational complexity is also proposed which uses the invariant property of a code. With these methods, the weight distributions of all extended binary primitive BCH codes of length 128 are computed, except for those for which the formulas of the weight distribution are known. It turns out that (128,64,22) extended binary primitive BCH code is formally self-dual. The probability of an undetectable error for each code is computed and its monotonicity is examined View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Constructions for difference triangle sets

    Page(s): 1346 - 1349
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (184 KB)  

    Difference triangle sets are useful in many practical problems of information transmission. This article studies combinatorial and computational constructions for difference triangle sets having small scopes. Our algorithms have been used to produce difference triangle sets whose scopes are the best currently known View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • The multicovering radii of codes

    Page(s): 1372 - 1377
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (264 KB)  

    The covering radius of a code is the least r such that the set of balls of radius r around codewords covers the entire ambient space. We introduce a generalization of the notion of covering radius. The m-covering radius of a code is the least radius such that the set of balls of that radius covers all m-tuples of elements in the ambient space. We investigate basic properties of m-covering radii. We investigate whether codes exist with given m-covering radii (not always). We derive bounds on the size of the smallest code with a given m-covering radius, based on generalizations of the sphere bound and the method of counting excesses View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Second-order noiseless source coding theorems

    Page(s): 1339 - 1341
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (132 KB)  

    Shannon's celebrated source coding theorem can be viewed as a “one-sided law of large numbers”. We formulate second-order noiseless source coding theorems for the deviation of the codeword lengths from the entropy. For a class of sources that includes Markov chains we prove a “one-sided central limit theorem” and a law of the iterated logarithm View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Computational power of neural networks: a characterization in terms of Kolmogorov complexity

    Page(s): 1175 - 1183
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (284 KB)  

    The computational power of recurrent neural networks is shown to depend ultimately on the complexity of the real constants (weights) of the network. The complexity, or information contents, of the weights is measured by a variant of resource-bounded Kolmogorov (1965) complexity, taking into account the time required for constructing the numbers. In particular, we reveal a full and proper hierarchy of nonuniform complexity classes associated with networks having weights of increasing Kolmogorov complexity View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Upper bounds on trellis complexity of lattices

    Page(s): 1294 - 1300
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (328 KB)  

    Unlike block codes, n-dimensional lattices can have minimal trellis diagrams with an arbitrarily large number of states, branches, and paths. In particular, we show by a counterexample that there is no f(n), a function of n, such that all rational lattices of dimension n have a trellis with less than f(n) states. Nevertheless, using a theorem due to Hermite, we prove that every integral lattice Λ of dimension n has a trellis T, such that the total number of paths in T is upper-bounded by P(T)⩽n!(2/√3)n2(n-1/2)V(Λ) n-1 where V(n) is the volume of Λ. Furthermore, the number of states at time i in T is upper-bounded by |Si|⩽(2/√3)i2(n-1)V(Λ)2i2 n/. Although these bounds are seldom tight, these are the first known general upper bounds on trellis complexity of lattices View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Randomized fusion rules can be optimal in distributed Neyman-Pearson detectors

    Page(s): 1281 - 1288
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (404 KB)  

    We show that randomized fusion rules can be locally optimal in distributed detection systems under the Neyman-Pearson criterion. This result is contrary to common belief. We first formulate conditions for a randomized fusion rule to be locally optimal. Then, we present distribution functions of local observations that satisfy these conditions View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Surfaces and the weight distribution of a family of codes

    Page(s): 1354 - 1360
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (340 KB)  

    We derive the weight distribution of the binary trace codes with words (Tr(axq+1+bx3+cx))x∈F*(q2) where a, b, c∈F(q2) and Tr is the trace map from F(q2) to F2. The weights of these words determine the exponential sums which were considered earlier by Moreno and Kumar (1994) and Lahtonen (1995). Results from the theory of quadratic forms play a role but the decisive argument is of an algebraic-geometric nature, namely, from the theory of surfaces View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • A general minimax result for relative entropy

    Page(s): 1276 - 1280
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (240 KB)  

    Suppose nature picks a probability measure Pθ on a complete separable metric space X at random from a measurable set P Θ={Pθ:θ∈Θ}. Then, without knowing θ, a statistician picks a measure Q on S. Finally, the statistician suffers a loss D(P0||Q), the relative entropy between Pθ and Q. We show that the minimax and maximin values of this game are always equal, and there is always a minimax strategy in the closure of the set of all Bayes strategies. This generalizes previous results of Gallager(1979), and Davisson and Leon-Garcia (1980) View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Almost sure convergence analysis of mixed time averages and kth-order cyclic statistics

    Page(s): 1265 - 1268
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (192 KB)  

    The strong law of large numbers for general processes is established under certain conditions and the convergence rate of time averages is given. These results are used to obtain the convergence rates and the almost sure convergence properties of mixed time averages and kth-order cyclic statistics View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Performance analysis of the matrix pair method for blind channel identification

    Page(s): 1245 - 1253
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (536 KB)  

    We study the estimation variance performance of the matrix pair (MP) method for estimating the impulse responses of multiple FIR channels driven by an unknown input sequence. A first-order perturbation analysis of the large-data-size performance of the MP method is presented and an explicit expression for the estimation variance is derived. Both the theoretical and simulation results are used to investigate the statistical performance of the MP method and a number of new insights are revealed View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Tight bounds on the minimum average weight per branch for rate (N-1)/N convolutional codes

    Page(s): 1301 - 1305
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (232 KB)  

    Consider a cycle in the state diagram of a convolutional code. The average weight per branch of the cycle is equal to the total Hamming weight of all labels on the branches divided by the number of branches. Let w0 be the minimum average weight per branch over all cycles in the state diagram, except the zero state self-loop of weight zero. Codes with low w0 result in high bit error probabilities when they are used with either Viterbi or sequential decoding. Hemmati and Costello (1980) showed that w0 is upper-bounded by 2ν-2/(3·2ν-2-1) for a class of (2,1) codes where ν denotes the constraint length. In the present correspondence it is shown that the bound is valid for a large class of (n,n-1) codes, n⩾2. Examples of high-rate codes with w0 equal to the upper bound are also given. Hemmati and Costello defined a class of codes to be asymptotically catastrophic if w 0 approaches zero for large ν. The class of (n,n-1) codes constructed by Wyner and Ash (1963) is shown to be asymptotically catastrophic. All codes in the class have minimum possible w0 equal to 1/ν View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Linear programming bounds for doubly-even self-dual codes

    Page(s): 1238 - 1244
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (448 KB)  

    Using a variant of the linear programming method we derive a new upper bound on the minimum distance d of doubly-even self-dual codes of length n. Asymptotically, for n growing, it gives d/n⩽0.166315···+o(1), thus improving on the Mallows-Odlyzko-Sloane bound of 1/6. To establish this, we prove that in any doubly even-self-dual code the distance distribution is asymptotically upper-bounded by the corresponding normalized binomial distribution in a certain interval View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Optimal phases for a family of quadriphase CDMA sequences

    Page(s): 1205 - 1217
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (624 KB)  

    Signature sequences with good even even and odd (or polyphase) correlations are crucial for asynchronous code-division multiple access (CDMA). When the data sequence is random, the even and odd (or polyphase) correlations are equally important. However, for most known signature sequences, only their even correlations were analyzed. It appears that determining the odd (or the polyphase) correlations is generally a very hard problem since the odd (or the polyphase) correlations depend on the phases of the signature sequences. Sole (1989), Boztas, Hammons, and Kumar (1992) found a family of quadriphase sequences that are asymptotically optimal. These sequences gain a factor √2 in terms of their maximum periodic even correlations when compared with the best possible binary phase-shift keying (BPSK) sequences. We find the optimal phases of these sequences. The optimality is in the sense that at these phases, the mean square values of the even, odd, and the polyphase correlations are minimal, and achieve the Welch (1974) bound-equality simultaneously. Furthermore, we show that at these phases, the average user interference of these sequences is always smaller than that of the ideal random signature sequences. Comprehensive analytical and numerical results show that good phase sequences can offer a nonnegligible amount of gain over bad phase sequences at modest and high signal-to-noise ratios View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Worst case additive noise for binary-input channels and zero-threshold detection under constraints of power and divergence

    Page(s): 1256 - 1264
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (336 KB)  

    Additive-noise channels with binary inputs and zero-threshold detection are considered. We study worst case noise under the criterion of maximum error probability with constraints on both power and divergence with respect to a given symmetric nominal noise distribution. Particular attention is focused on the cases of a) Gaussian nominal distributions and b) asymptotic increase in worst case error probability when the divergence tolerance tends to zero View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Bidirectional sequential decoding

    Page(s): 1319 - 1326
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (272 KB)  

    The main drawback of sequential decoding is the variability of its decoding effort which could cause decoding erasures. We propose and analyze an efficient bidirectional sequential decoding (BSD) technique to alleviate this drawback. In the proposed BSD, two decoders are used; one is called a forward decoder (FD), and is used to search the tree from the forward direction; while the other is called a backward decoder (BD), and is used for the backward search of the tree. Forward decoding and backward decoding are performed simultaneously, and stop whenever the FD and BD merge at a common encoder state somewhere in the tree. The relationships between backward coding and forward coding are examined in detail. Good rate 1/2 convolutional codes, with memory m ranging from 2 to 25, suitable for bidirectional decoding found through extensive computer search, are provided. These codes possess the same distance properties from both forward and backward directions. It is found, by means of extensive computer simulations as well as a heuristic argument, that the advantage of the BSD appears as a substantial decrease of the computational variability of the sequential decoding. Our findings suggest that the Pareto exponent of unidirectional sequential decoding (USD) can be practically doubled by using BSD View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Complexity-constrained trellis quantizers

    Page(s): 1134 - 1144
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (604 KB)  

    A reasonable measure of quantizer complexity is the expected number of quanta per input sample for which distortion is computed. Given this measure, a rate-distortion-complexity theory is obtained by extending earlier work in alphabet-constrained rate-distortion theory. Numerical results show that operation on the alphabet-constrained rate-distortion bound can be obtained with a complexity of two. Furthermore, Lloyd-Max conditions are shown to describe the minimum of a slightly constrained version of the rate-distortion-complexity problem. Complexity-constrained design methods are applied first to trellis-coded quantizers, where they are shown to reduce arithmetic operations by at least 25%. They are then used to develop model-based trellis quantizers, the trellises of which are derived from a Markov model of the source. Simulation results confirm that excellent performance can be obtained with modest complexity View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Contribution to Munuera's problem on the main conjecture of geometric hyperelliptic MDS codes

    Page(s): 1349 - 1354
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (280 KB)  

    In coding theory, it is of great interest to know the maximal length of MDS codes. In fact, the main conjecture says that the length of MDS codes over Fq is less than or equal to q+1 (except for some special cases). Munuera (see ibid., vol.38, p.1573-7, 1992) proposed a new way to attack the main conjecture on MDS codes for geometric codes. In particular, he proved the conjecture for codes arising from curves of genus one or two when the cardinal of the ground field is large enough. He also asked whether a similar theorem can be proved for any hyperelliptic curve. The purpose of this correspondence is to give an affirmative answer. In fact, our method also proves the main conjecture for geometric MDS codes for q=2 if the genus of the hyperelliptic curve is either 1, 2 or 3, and for q=3 if the genus of the curve is 1 View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • About distances of discrete distributions satisfying the data processing theorem of information theory

    Page(s): 1288 - 1293
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (224 KB)  

    The distances of discrete probability distributions are considered. Necessary and sufficient conditions for validity of the data processing theorem of information theory are established. These conditions are applied to the Burbea-Rao (1982) divergences and Bregman (1967) distances View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Robust and efficient recovery of a signal passed through a filter and then contaminated by non-Gaussian noise

    Page(s): 1184 - 1191
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (356 KB)  

    Consider a channel where a continuous periodic input signal is passed through a linear filter and then is contaminated by an additive noise. The problem is to recover this signal when we observe n repeated realizations of the output signal. Adaptive efficient procedures, that are asymptotically minimax over all possible procedures, are known for channels with Gaussian noise and no filter (the case of direct observation). Efficient procedures, based on the smoothness of a recovered signal, are known for the case of Gaussian noise. Robust rate-optimal procedures are known as well. However, there are no results on robust and efficient data-driven procedures; moreover, the known results for the case of direct observation indicate that even a small deviation from Gaussian noise may lead to a drastic change. We show that for the considered case of indirect data and a particular class of so-called supersmooth filters there exists a procedure of recovery of an input signal that possesses the desired properties; namely, it is: adaptive to the smoothness of the input signal; robust to the distribution of the noise; globally and pointwise-efficient, that is, its minimax global and pointwise risks converge with the best constant and rate over all possible estimators as n→∞; and universal in the sense that for a wide class of linear (not necessarily bounded) operators the efficient estimator is a plug-in one. Furthermore, we explain how to employ the obtained asymptotic results for the practically important case of small n (large noise) View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.

Aims & Scope

IEEE Transactions on Information Theory publishes papers concerned with the transmission, processing, and utilization of information.

Full Aims & Scope

Meet Our Editors

Editor-in-Chief
Frank R. Kschischang

Department of Electrical and Computer Engineering