By Topic

IEEE Transactions on Information Theory

Issue 7 • July 2004

Filter Results

Displaying Results 1 - 22 of 22
  • Table of contents

    Publication Year: 2004, Page(s): c1
    Request permission for commercial reuse | PDF file iconPDF (33 KB)
    Freely Available from IEEE
  • IEEE Transactions on Information Theory publication information

    Publication Year: 2004, Page(s): c2
    Request permission for commercial reuse | PDF file iconPDF (38 KB)
    Freely Available from IEEE
  • Problems on Sequences: Information Theory and Computer Science Interface

    Publication Year: 2004, Page(s):1385 - 1392
    Cited by:  Papers (4)
    Request permission for commercial reuse | PDF file iconPDF (224 KB) | HTML iconHTML
    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Markov types and minimax redundancy for Markov sources

    Publication Year: 2004, Page(s):1393 - 1402
    Cited by:  Papers (26)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (312 KB) | HTML iconHTML

    Redundancy of universal codes for a class of sources determines by how much the actual code length exceeds the optimal code length. In the minimax scenario, one designs the best code for the worst source within the class. Such minimax redundancy comes in two flavors: average minimax or worst case minimax. We study the worst case minimax redundancy of universal block codes for Markovian sources of ... View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Power laws for monkeys typing randomly: the case of unequal probabilities

    Publication Year: 2004, Page(s):1403 - 1414
    Cited by:  Papers (7)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (320 KB) | HTML iconHTML

    An early result in the history of power laws, due to Miller, concerned the following experiment. A monkey types randomly on a keyboard with N letters (N>1) and a space bar, where a space separates words. A space is hit with probability p; all other letters are hit with equal probability (1-p)/N. Miller proved that in this experiment, the rank-frequency distribution of words follows a power law. Th... View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Grammar-based lossless universal refinement source coding

    Publication Year: 2004, Page(s):1415 - 1424
    Cited by:  Papers (5)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (280 KB) | HTML iconHTML

    A sequence y=(y1,...,yn) is said to be a coarsening of a given finite-alphabet source sequence x=(x1,...,xn) if, for some function φ, yi=φ(xi) (i=1,...,n). In lossless refinement source coding, it is assumed that the decoder already possesses a coarsening y of a given source sequence x. It is the job of the lossless refineme... View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Compression of words over a partially commutative alphabet

    Publication Year: 2004, Page(s):1425 - 1441
    Cited by:  Papers (5)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (376 KB) | HTML iconHTML

    Concurrency is a fundamental concept in computer science which is concerned with the study of systems involving multiple processes. The order of events in a concurrent system is unpredictable because of the independence of events occurring in the individual processes. Trace theory is a successful model for the execution of concurrent processes which employs congruence classes of words over partial... View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Linear time universal coding and time reversal of tree sources via FSM closure

    Publication Year: 2004, Page(s):1442 - 1468
    Cited by:  Papers (37)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (728 KB) | HTML iconHTML

    Tree models are efficient parametrizations of finite-memory processes, offering potentially significant model cost savings. The information theory literature has focused mostly on redundancy aspects of the universal estimation and coding of these models. In this paper, we investigate representations and supporting data structures for finite-memory processes, as well as the major impact these struc... View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Universal compression of memoryless sources over unknown alphabets

    Publication Year: 2004, Page(s):1469 - 1481
    Cited by:  Papers (65)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (328 KB) | HTML iconHTML

    It has long been known that the compression redundancy of independent and identically distributed (i.i.d.) strings increases to infinity as the alphabet size grows. It is also apparent that any string can be described by separately conveying its symbols, and its pattern-the order in which the symbols appear. Concentrating on the latter, we show that the patterns of i.i.d. strings over all, includi... View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • On the efficient evaluation of probabilistic similarity functions for image retrieval

    Publication Year: 2004, Page(s):1482 - 1496
    Cited by:  Papers (49)  |  Patents (3)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (608 KB) | HTML iconHTML

    Probabilistic approaches are a promising solution to the image retrieval problem that, when compared to standard retrieval methods, can lead to a significant gain in retrieval accuracy. However, this occurs at the cost of a significant increase in computational complexity. In fact, closed-form solutions for probabilistic retrieval are currently available only for simple probabilistic models such a... View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Some stochastic properties of memoryless individual sequences

    Publication Year: 2004, Page(s):1497 - 1505
    Cited by:  Papers (2)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (232 KB) | HTML iconHTML

    An individual sequence of real numbers is memoryless if no continuous Markov prediction scheme of finite order can outperform the best constant predictor under the squared loss. It is established that memoryless sequences satisfy an elementary law of large numbers, and sliding-block versions of Hoeffding's inequality and the central limit theorem. It is further established that memoryless binary s... View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Finite-memory universal prediction of individual sequences

    Publication Year: 2004, Page(s):1506 - 1523
    Cited by:  Papers (8)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (448 KB) | HTML iconHTML

    The problem of predicting the next outcome of an individual binary sequence under the constraint that the universal predictor has a finite memory, is explored. In this analysis, the finite-memory universal predictors are either deterministic or random time-invariant finite-state (FS) machines with K states (K-state machines). The paper provides bounds on the asymptotic achievable regret of these c... View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Performance analysis of grammar-based codes revisited

    Publication Year: 2004, Page(s):1524 - 1535
    Cited by:  Papers (3)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (552 KB) | HTML iconHTML

    The compression performance of grammar-based codes is revisited from a new perspective. Previously, the compression performance of grammar-based codes was evaluated against that of the best arithmetic coding algorithm with finite contexts. In this correspondence, we first define semifinite-state sources and finite-order semi-Markov sources. Based on the definitions of semifinite-state sources and ... View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • On the hardness of finding optimal multiple preset dictionaries

    Publication Year: 2004, Page(s):1536 - 1539
    Cited by:  Papers (3)  |  Patents (1)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (144 KB) | HTML iconHTML

    We show that the following simple compression problem is NP-hard: given a collection of documents, find the pair of Huffman dictionaries that minimizes the total compressed size of the collection, where the best dictionary from the pair is used to compress each document. We also show the NP-hardness of finding optimal multiple preset dictionaries for LZ'77-based compression schemes. Our reductions... View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Monotonicity-based fast algorithms for MAP estimation of Markov sequences over noisy channels

    Publication Year: 2004, Page(s):1539 - 1544
    Cited by:  Papers (19)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (248 KB) | HTML iconHTML

    In this correspondence, we study algorithmic approach to solving the problem of maximum a posteriori (MAP) estimation of Markov sequences transmitted over noisy channels, which is also known as the MAP decoding problem. For the class of memoryless binary channels that produce independent substitution and erasure errors, the MAP sequence estimation problem can be formulated and solved as one of the... View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Shared information and program plagiarism detection

    Publication Year: 2004, Page(s):1545 - 1551
    Cited by:  Papers (79)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (328 KB) | HTML iconHTML

    A fundamental question in information theory and in computer science is how to measure similarity or the amount of shared information between two sequences. We have proposed a metric, based on Kolmogorov complexity, to answer this question and have proven it to be universal. We apply this metric in measuring the amount of shared information between two computer programs, to enable plagiarism detec... View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Universal entropy estimation via block sorting

    Publication Year: 2004, Page(s):1551 - 1561
    Cited by:  Papers (27)
    Request permission for commercial reuse | Click to expandAbstract | PDF file iconPDF (384 KB) | HTML iconHTML

    In this correspondence, we present a new universal entropy estimator for stationary ergodic sources, prove almost sure convergence, and establish an upper bound on the convergence rate for finite-alphabet finite memory sources. The algorithm is motivated by data compression using the Burrows-Wheeler block sorting transform (BWT). By exploiting the property that the BWT output sequence is close to ... View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Contributors

    Publication Year: 2004, Page(s):1562 - 1565
    Cited by:  Papers (1)
    Request permission for commercial reuse | PDF file iconPDF (61 KB)
    Freely Available from IEEE
  • 2005 IEEE International Symposium on Information Theory

    Publication Year: 2004, Page(s): 1566
    Request permission for commercial reuse | PDF file iconPDF (280 KB)
    Freely Available from IEEE
  • 2004 IEEE Membership Application

    Publication Year: 2004, Page(s):1567 - 1568
    Request permission for commercial reuse | PDF file iconPDF (744 KB)
    Freely Available from IEEE
  • IEEE Transactions on Information Theory information for authors

    Publication Year: 2004, Page(s): c3
    Request permission for commercial reuse | PDF file iconPDF (23 KB) | HTML iconHTML
    Freely Available from IEEE
  • Blank page [back cover]

    Publication Year: 2004, Page(s): c4
    Request permission for commercial reuse | PDF file iconPDF (2 KB)
    Freely Available from IEEE

Aims & Scope

IEEE Transactions on Information Theory publishes papers concerned with the transmission, processing, and utilization of information.

Full Aims & Scope

Meet Our Editors

Editor-in-Chief
Frank R. Kschischang

Department of Electrical and Computer Engineering