Information Theory, IRE Transactions on
This Transactions ceased production in 1962. The current retitled publication is IEEE Transactions on Information Theory.
Latest Published Articles

Multiple error correction by means of parity checks
Feb18 2014 
A Markoff envelope process
Feb18 2014 
On signal parameter estimation
Feb18 2014 
The utility of a communication channel and applications to suboptimal information handling procedures
Feb18 2014 
Time statistics of noise
Feb18 2014
Popular Articles

Visual pattern recognition by moment invariants
Jan06 2003 
Lowdensity paritycheck codes
Jan06 2003 
A statistical theory of target detection by pulsed radar
Jan06 2003 
An introduction to matched filters
Jan06 2003 
The zero error capacity of a noisy channel
Jan06 2003
Publish in this Journal
Popular Articles (April 2015)
Includes the top 50 most frequently downloaded documents for this publication according to the most recent monthly usage statistics.
1. Visual pattern recognition by moment invariants
Publication Year: 1962 , Page(s): 179  187
Cited by: Papers (1234)  Patents (30)In this paper a theory of twodimensional moment invariants for planar geometric figures is presented. A fundamental theorem is established to relate such moment invariants to the wellknown algebraic invariants. Complete systems of moment invariants under translation, similitude and orthogonal transformations are derived. Some moment invariants under general twodimensional linear transformations are also included. Both theoretical formulation and practical models of visual pattern recognition based upon these moment invariants are discussed. A simple simulation program together with its performance are also presented. It is shown that recognition of geometrical patterns and alphabetical characters independently of position, size and orientation can be accomplished. It is also indicated that generalization is possible to include invariance with parallel projection. View full abstract»

2. Lowdensity paritycheck codes
Publication Year: 1962 , Page(s): 21  28
Cited by: Papers (1646)  Patents (203)A lowdensity paritycheck code is a code specified by a paritycheck matrix with the following properties: each column contains a small fixed number of l's and each row contains a small fixed number of l's. The typical minimum distance of these codes increases linearly with block length for a fixed rate and fixed . When used with maximum likelihood decoding on a sufficiently quiet binaryinput symmetric channel, the typical probability of decoding error decreases exponentially with block length for a fixed rate and fixed . A simple but nonoptimum decoding scheme operating directly from the channel a posteriori probabilities is described. Both the equipment complexity and the datahandling capacity in bits per second of this decoder increase approximately linearly with block length. For and a sufficiently low rate, the probability of error using this decoder on a binary symmetric channel is shown to decrease at least exponentially with a root of the block length. Some experimental results show that the actual probability of decoding error is much smaller than this theoretical bound. View full abstract»

3. A statistical theory of target detection by pulsed radar
Publication Year: 1960 , Page(s): 59  267
Cited by: Papers (263)This report presents data from which one may obtain the probability that a pulsedtype radar system will detect a given target at any range. This is in contrast to the usual method of obtaining radar range as a single number, which can be taken mathematically to imply that the probability of detection is zero at any range greater than this number, and certainty at any range less than this number. Three variables, which have so far received little quantitative attention in the subject of radar range, are introduced in the theory: l.The time taken to detect the target. 2.The average time interval between false alarms (false indications of targets). 3.The number of pulses integrated. It is shown briefly how the results for pulsedtype systems may be applied in the case of continuouswave systems. Those concerned with systems analysis problems including radar performance may profitably use this work as one link in a chain involving several probabilities. For instance, the probability that a given aircraft will be detected at least once while flying any given path through a specified model radar network may be calculated using the data presented here as a basis, provided that additional probability data on such things as outage time etc., are available. The theory developed here does not take account of interference such as clutter or manmade static, but assumes only random noise at the receiver input. Also, an ideal type of electronic integrator and detector are assumed. Thus the results are the best that can be obtained under ideal conditions. It is not too difficult, however, to make reasonable assumptions which will permit application of the results to the currently available types of radar. The first part of this report is a restatement of known radar fundamentals and supplies continuity and background for what follows. The mathematical part of the theory is not contained herein, but will be issued subsequently as a Separate report(2a) View full abstract»

4. An introduction to matched filters
Publication Year: 1960 , Page(s): 311  329
Cited by: Papers (192)  Patents (12)In a tutorial exposition, the following topics are discussed: definition of a matched filter; where matched filters arise; properties of matched filters; matchedfilter synthesis and signal specification; some forms of matched filters. View full abstract»

5. The zero error capacity of a noisy channel
Publication Year: 1956 , Page(s): 8  19
Cited by: Papers (283)The zero error capacity of a noisy channel is defined as the least upper bound of rates at which it is possible to transmit information with zero probability of error. Various properties of are studied; upper and lower bounds and methods of evaluation of are given. Inequalities are obtained for the relating to the "sum" and "product" of two given channels. The analogous problem of zero error capacity for a channel with a feedback link is considered. It is shown that while the ordinary capacity of a memoryless channel with feedback is equal to that of the same channel without feedback, the zero error capacity may be greater. A solution is given to the problem of evaluating . View full abstract»

6. Probability of detection for fluctuating targets
Publication Year: 1960 , Page(s): 269  308
Cited by: Papers (145)This report considers the probability of detection off a target by a pulsed search radar, when the target has a fluctuating cross section. Formulas for detection probability are derived, and curves off detection probability vs, range are given, for four different target fluctuation models. The investigation shows that, for these fluctuation models, the probability of detection for a fluctuating target is less than that for a nonfluctuating target if the range is sufficiently short, and is greater if the range is sufficiently long. The amount by which the fluctuating and nonfluctuating cases differ depends on the rapidity of fluctuation and on the statistical distribution of the fluctuations. Figure 18, p. 307, shows a comparison between the nonfluctuating case and the four fluctuating cases considered. View full abstract»

7. Three models for the description of language
Publication Year: 1956 , Page(s): 113  124
Cited by: Papers (132)  Patents (8)We investigate several conceptions of linguistic structure to determine whether or not they can provide simple and "revealing" grammars that generate all of the sentences of English and only these. We find that no finitestate Markov process that produces symbols with transition from state to state can serve as an English grammar. Furthermore, the particular subclass of such processes that produce order statistical approximations to English do not come closer, with increasing , to matching the output of an English grammar. We formalizethe notions of "phrase structure" and show that this gives us a method for describing language which is essentially more powerful, though still representable as a rather elementary type of finitestate process. Nevertheless, it is successful only when limited to a small subset of simple sentences. We study the formal properties of a set of grammatical transformations that carry sentences with phrase structure into new sentences with derived phrase structure, showing that transformational grammars are processes of the same elementary type as phrasestructure grammars; that the grammar of English is materially simplified if phrase structure description is limited to a kernel of simple sentences from which all other sentences are constructed by repeated transformations; and that this view of linguistic structure gives a certain insight into the use and understanding of language. View full abstract»

8. A useful theorem for nonlinear devices having Gaussian inputs
Publication Year: 1958 , Page(s): 69  72
Cited by: Papers (159)  Patents (1)If and only if the inputs to a set of nonlinear, zeromemory devices are variates drawn from a Gaussian random process, a useful general relationship may be found between certain input and output statistics of the set. This relationship equates partial derivatives of the (highorder) output correlation coefficient taken with respect to the input correlation coefficients, to the output correlation coefficient of a new set of nonlinear devices bearing a simple derivative relation to the original set. Application is made to the interesting special cases of conventional crosscorrelation and autocorrelation functions, and Bussgang's theorem is easily proved. As examples, the output autocorrelation functions are simply obtained for a hard limiter, linear detector, clipper, and smooth limiter. View full abstract»

9. Phase shift pulse codes with good periodic correlation properties (Corresp.)
Publication Year: 1962 , Page(s): 381  382
Cited by: Papers (169)  Patents (22)First Page of the ArticleView full abstract» 
10. Complementary series
Publication Year: 1961 , Page(s): 82  87
Cited by: Papers (407)  Patents (68)A set of complementary series is defined as a pair of equally long, finite sequences of two kinds of elements which have the property that the number of pairs of like elements with any one given separation in one series is equal to the number of pairs of unlike elements with the same given separation in the other series. (For instance the two series, 1001010001 and 1000000110 have, respectively, three pairs of like and three pairs of unlike adjacent elements, four pairs of like and four pairs of unlike alternate elements, and so forth for all possible separations.) These series, which were originally conceived in connection with the optical problem of multislit spectrometry, also have possible applications in communication engineering, for when the two kinds of elements of these series are taken to be +1 and 1, it follows immediately from their definition that the sum of their two respective autocorrelation series is zero everywhere, except for the center term. Several propositions relative to these series, to their permissible number of elements, and to their synthesis are demonstrated. View full abstract»

11. Quantizing for minimum distortion
Publication Year: 1960 , Page(s): 7  12
Cited by: Papers (725)  Patents (51)This paper discusses the problem of the minimization of the distortion of a signal by a quantizer when the number of output levels of the quantizer is fixed. The distortion is defined as the expected value of some function of the error between the input and the output of the quantizer. Equations are derived for the parameters of a quantizer with minimum distortion. The equations are not soluble without recourse to numerical methods, so an algorithm is developed to simplify their numerical solution. The case of an input signal with normally distributed amplitude and an expected squared error distortion measure is explicitly computed and values of the optimum quantizer parameters are tabulated. The optimization of a quantizer subject to the restriction that both input and output levels be equally spaced is also treated, and appropriate parameters are tabulated for the same case as above. View full abstract»

12. Phase shift pulse codes with good periodic correlation properties
Publication Year: 1961 , Page(s): 254  257
Cited by: Papers (42)  Patents (5)A method of generating phase shift pulse codes of arbitrarily long length with zero periodic correlation except for the peak for zero shift is presented. The codes are of length where is any prime number, and different phase shifts corresponding to the th roots of unity are necessary to generate them. Since different phase shifts are required, these codes are not as easy to generate and process as the binary codes, but this does not seem to be a serious limitation to their usefulness. Application of these codes can be made as interpulse phase modulation for range resolution in pulse Doppler radars or for a method of synchronizing a pulse code communication system. View full abstract»

13. Human memory and the storage of information
Publication Year: 1956 , Page(s): 129  137
Cited by: Papers (9)The amount of selective information in a message can be increased either by increasing the variety of the symbols from which it is composed or by increasing the length of the message. Psychological experiments indicate that the variety of the symbols is far less important than the length of the message in controlling what human subjects are able to remember. Two messages equal in length but differing in the amount of information per symbol are equally easy to memorize. This fact provides an opportunity for the effective use of recoding procedures and reveals the mental economy involved in organizing the materials we want to remember. An apparent exception to the rule that length, not variety, is the limiting factor in human memory occurs in the case of redundant messages. If two messages of the same length differ because one contains redundancy familiar to the learner and the other does not, the redundant message will usually be easier to learn and remember. In terms of the theory of information, redundancy can be viewed equally well as a reduction in the information per symbol or as a reduction in the effective length of the message. Psychologically, however, these two alternatives are not equivalent; redundancy permits a reorganization into familiar sequences in a way that effectively shortens the length of the message and so makes it easier to memorize, but this is not psychologically equivalent to reducing the amount of information per symbol. It is as if each storage register could accept any one of a tremendous variety of alternative symbols, but the number of registers available was quite limited. If we use these registers to store binary symbols, the storage is inefficient. If we group the binary symbols into sequences, give each sequence a different name, and store the recoded names, we can make much more efficient use of the registers. Familiar redundancy is helpful because it enables us to recode more efficiently. These results for human memory are all th e more striking in view of the fact that the amount of information per symbol is a critically important variable controlling the accuracy of our perceptions. View full abstract»

14. Decision units in the perception of speech
Publication Year: 1962 , Page(s): 81  83
Cited by: Papers (9)  Patents (1)It has been shown experimentally that speech intelligibility is a function of grammatical content. This fact implies that automatic speech recognizers may well need to include information about linguistic structure. View full abstract»

15. Picture coding using pseudorandom noise
Publication Year: 1962 , Page(s): 145  154
Cited by: Papers (98)  Patents (141)In order to transmit television pictures over a digital channel, it is necessary to send a binary code which represents the intensity level at each point in the picture. For good picture quality using standard PCM transmission, at least six bits are required at each sample point, since the eye is very sensitive to the small intensity steps introduced by quantization. However, by simply adding some noise to the signal before it is quantized and subtracting the same noise at the receiver, the quantization steps can be broken up and the source rate reduced to three bits per sample. Pseudorandom number generators can be synchronized at the transmitter and receiver to provide the identical "noise" which makes the process possible. Thus, with the addition of only a small amount of equipment, the efficiency of a PCM channel can be doubled. View full abstract»

16. Design factors in the development of an optical character recognition machine
Publication Year: 1962 , Page(s): 167  171
Cited by: Papers (4)  Patents (3)Interest in optical character recognition has grown tremendously during the past several years. When one considers the volume of printed information that must be accessible both to human readers and to data processing machines, the increasing interest is not at all surprising. Many optical character recognition principles that appear to be quite powerful are impractical or too costly to implement with available technology. Other principles that appear relatively economical require an input quality that is impractical to generate. The field for commercial development falls somewhere between these two extremes. Many factors determine the character recognition principles to be utilized in the development of a machine system. Of primary importance are: 1) The shapes of the symbols to be sensed, 2) The number of different symbols to be discriminated, 3) The print quality range that must be accommodated, and 4) The required performance of the system. Unfortunately, in the present state of the art, the relationship of these factors to each other and to a given principle can be only qualitatively defined. This paper describes and discusses the development of a practical optical character recognition system, the character recognition portion of the IBM 1418 Optical Character Reader. The considerations involved are applicable to the development of any character recognition system. View full abstract»

17. On a moment theorem for complex Gaussian processes
Publication Year: 1962 , Page(s): 194  195
Cited by: Papers (97)A general theorem is provided for the moments of a complex Gaussian video process. This theorem is analogous to the wellknown property of the multivariate normal distribution for real variables, which states that an th order central product moment is zero if is odd and is equal to a sum of products of covariances when is even. View full abstract»

18. Encoding and errorcorrection procedures for the BoseChaudhuri codes
Publication Year: 1960 , Page(s): 459  470
Cited by: Papers (65)  Patents (1)Bose and RayChaudhuri have recently described a class of binary codes which for arbitrary and are error correcting and have length of which no more than digits are redundancy. This paper describes a simple errorcorrection procedure for these codes. Their cyclic structure is demonstrated and methods of exploiting it to implement the coding and correction procedure using shift registers are outlined. Closer bounds on the number of redundancy digits are derived. View full abstract»

19. A new upper bound for errorcorrecting codes
Publication Year: 1962 , Page(s): 203  207
Cited by: Papers (94)By refining Hamming's geometric spherepacking model a new upper bound for nonsystematic binary errorcorrecting codes is found. Only combinatorial arguments are used. Whereas Hamming's upper bound estimate for errorcorrecting codes involved a count of all points Hamming distance from the set of code points, the model is extended here to include consideration of points which are distance away from the code set. The percentage improvement from Hamming's bounds is sometimes quite sizable for cases of two or more errors to be corrected. The new bound improves on Wax's bounds in all but four of the cases he lists. View full abstract»

20. Design and performance of phaselock circuits capable of nearoptimum performance over a wide range of input signal and noise levels
Publication Year: 1955 , Page(s): 66  76
Cited by: Papers (73)Phase locked loops (PLLs) provide an efficient method for detection and tracking of narrowband signals in the presence of wideband noise. This paper explains how minimumrmserror loops may be designed if the inputsignal level, inputnoise level, and a specification for transient performance are given. However, the system performance of PLLs departs rapidly from the best obtainable performance if either the signal or the noise levels are different from the design levels, and if no compensating changes are made in the PLL. A marked improvement results if the total input power is held constant, regardless of signal or noise levels. It will be demonstrated that a fixedcomponent loop preceded by a bandpass limiter yields nearoptimum performance over a wide range of input signal and noise levels. The following topics are discussed: 1. An outline of the theoretical design of minimmum rmserror, PLLs when inputsignal level, inputnoise level, and a specification for transient error are given. 2. The effects of different input levels of signal and noise: a. On a system having a fixedcomponent loop that is optimum only for an original set of levels. b. On a system in which loop components maintain optimum performance when the new levels are given. 3. Characteristics of a bandpass limiter. 4. A comparison of the effect of different signal and noise levels: a. On a loop using a fixed filter preceded by an automatic gain control (AGC) system that holds the signal level constant. b. On a fixedfilter loop preceded by a bandpass limiter. c. On a variablefilter loop continually adjusted to be optimum. 5. Experimental verification of the fixedcomponent loop preceded by a bandpass limiter. View full abstract»

21. Visual Pattern Discrimination
Publication Year: 1962 , Page(s): 84  92
Cited by: Papers (117)  Patents (2)Visual discrimination experiments were conducted using unfamiliar displays generated by a digital computer. The displays contained two sidebyside fields with different statistical, topological or heuristic properties. Discrimination was defined as that spontaneous visual process which gives the immediate impression of two distinct fields. The condition for such discrimination was found to be based primarily on clusters or lines formed by proximate points of uniform brightness. A similar rule of connectivity with hue replacing brightness was obtained by using varicolored dots of equal subjective brightness. The limitations in discriminating complex line structures were also investigated. View full abstract»

22. Envelopes and preenvelopes of real waveforms
Publication Year: 1958 , Page(s): 53  57
Cited by: Papers (89)  Patents (4)Rice's formula for the "envelope" of a given signal is very cumbersome; in any case where the signal is not a single sine wave, the analytical use and explicit calculation of the envelope is practically prohibitive. A different formula for the envelope is given herein which is much simpler and easier to handle analytically. We show precisely that if is the Hilbert transform of , then Rice's envelope of is the absolute value of the complexvalued function . The function is called the preenvelope of and is shown to be involved implicitly in some other usual engineering practices. The Hilbert transform is then studied; it is shown that has the same power spectrum as and is uncorrelated with at the same time instant. Further, the autocorrelation of the preenvelope of is twice the preenvelope of the autocorrelation of . By using the preenvelope, the envelope of the output of a linear filter is easily calculated, and this is used to compute the first probability density for the envelope of the output of an arbitrary linear filter when the input is an arbitrary signal plus Gaussian noise. An application of preenvelopes to the frequency modulation of an arbitrary waveform by another arbitrary waveform is also given. View full abstract»

23. Binary codes with specified minimum distance
Publication Year: 1960 , Page(s): 445  450
Cited by: Papers (48)Two digit sequences, called "points," of binary digits are said to be at distance if exactly corresponding digits are unlike in the two sequences. The construction of sets of points, called codes, in which some specified minimum distance is maintained between pairs of points is of interest in the design of selfchecking systems for computing with or transmitting binary digits, the minimum distance being the minimum number of digital errors required to produce an undetected error in the system output. Previous work in the field had established general upper bounds for the number of digit points in codes of minimum distance d with certain properties. This paper gives new results in the field in the form of theorems which permit systematic construction of codes for given ; for some , the codes contain the greatest possible numbers of points. View full abstract»

24. On the detection of stochastic signals in additive normal noiseI
Publication Year: 1957 , Page(s): 86  121
Cited by: Papers (32)  Patents (1)The problem of optimum and suboptimum detection of normal signals in additive normal noise backgrounds is examined by the methods of statistical decision theory. Some general results for optimum receiver structure, error probabilities, and average risk are obtained for the case of colored noise backgrounds. A detailed study of threshold reception in whitenoise backgrounds is included, along with calculations of Bayes risk, bias terms, and minimum detectable signals for broadband RCnoise signals and narrowband, i.e., highQ, LRCnoise signal processes. Optimum detector structures for signal processes with rational intensity spectra are also determined for the white noise case, and particular attention is paid to optimum receiver design in terms of physically realizable elements. Suboptimum receiver structure and performance are considered briefly, as well as a number of limiting cases of more special interest. General methods of attack are illustrated, with details given in Appendixes IV. Application of the results to a variety of communication problems is indicated. View full abstract»

25. On the asmptotic efficiency of locally optimum detectors
Publication Year: 1961 , Page(s): 67  71
Cited by: Papers (59)A detector examines an unknown waveform to determine whether it is a mixture of signal and noise, or noise alone. The NeymanPearson detector is optimum in the sense that for given false alarm probability, signaltonoise ratio, and number of observations, it minimizes the false dismissal probability. This detector is optimum for all values of the signaltonoise ratio, and its implementation is usually quite complicated. In many situations it is desired to detect signals which are very weak compared to the noise. The locally optimum detector is defined as one which has optimum properties only for small signaltonoise ratios. It is proposed as an alternative to the NeymanPearson detector, since in practice it is usually only necessary to have a nearoptimum detector for weak signals, since strong signals will be detected with reasonable accuracy even if the detector is well below optimum. In order to evaluate the performance of the locally optimum detector, it is compared to the NeymanPearson detector. This comparison is based on the concept of asymptotic relative efficiency introduced by Pitman for comparing hypothesis testing procedures. On the basis of this comparison, it is shown that the locally optimum detector is asymptotically as efficient as the NeymanPearson detector. A number of applications to several detection problems are considered. It is found that the implementation of the locally optimum detector is less, or at most as complicated as that of the NeymanPearson detector. View full abstract»

26. Error correcting codes and their implementation for data transmission systems
Publication Year: 1961 , Page(s): 234  244
Cited by: Papers (21)Presented here is a practical automatic errorcorrecting system that may be applied to many data transmission problems. It is particularly suited to the correction of bursts of errors and so may be applied to the problem of the transmission of data over telephone networks. The attractive feature of the system is its remarkable simplicity from the point of view of implementation. It is so simple that it can readily be incorporated into much existing equipment. In the system, messages are transmitted in blocks and each block is coded separately. The codes used within the blocks are cyclic codes. This means that coders and decoders employ linear feedback shift registers to form check digits and to correct errors. The basic ideas are presented in terms of the hardware components to which the system gives rise and analyzed afterwards in terms of mathematics so that it is easy for the engineer to see, at once, what is involved. The theory usually applies to binary messages in which data is transmitted serially. However, an extension is included which shows how the same ideas may be applied to binary codes in which information is sent in parallel. View full abstract»

27. A new interpretation of information rate
Publication Year: 1956 , Page(s): 185  189
Cited by: Papers (43)If the input symbols to a communication channel represent the outcomes of a chance event on which bets are available at odds consistent with their probabilities (i.e., "fair" odds), a gambler can use the knowledge given him by the received symbols to cause his money to grow exponentially. The maximum exponential rate of growth of the gambler's capital is equal to the rate of transmission of information over the channel. This result is generalized to include the case of arbitrary odds. Thus we find a situation in which the transmission rate has significance even though no coding is contemplated. Previously this quantity was given significance only by a theorem of Shannon's which asserted that, with suitable encoding, binary digits could be transmitted over the channel at this rate with an arbitrarily small probability of error. View full abstract»

28. On the problem of time jitter in sampling
Publication Year: 1962 , Page(s): 226  236
Cited by: Papers (55)  Patents (2)There are many communication as well as control systemsin fact, an increasing number of themin which at some stage a continuous data source is sampled "periodically," at the nominal rate of samples a second, being the highest frequency component of the data. There are, generally speaking, two kinds of errors introduced by the sampling mechanism: errors in amplitude and errors in timing, or "time jitter." This paper is concerned with the latter. We assume a random model for the jitter. We begin with a study of the properties of the jittered samples for both deterministic and stochastic signals. Depending on the stochastic properties of the jitter, the presence of a discrete component in the signal may give rise to new discrete components as a result of jitter. Generally speaking, however, the effect of jitter is to produce a (frequency) selective attenuation as well as a uniform spectral density component. The more correlation in the jitter, the less the spectral distribution is affected. Various measures of the "error" due to jitter are estimated. Thus the error may be the mean square in the fitted samples or some linear or nonlinear operation thereof. Also, weighted meansquare errors are considered, and general methods of estimating these errors are developed. The problem of optimal use of the jittered samples is next considered. Interpreting the optimality to be in the meansquare sense, an explicit solution for optimal linear operation is obtained. Also for a wide class of signals it is shown that jitter does not affect the nature of the optimal operations; linear operations, for instance, remain linear, although with different weights. To illustrate the methods an example drawn from telemetry is given, where the timing is derived from the zero crossings of a sine wave and the time jitter is taken as due to noise. The jitter is highly correlated and the results involve some lengthy calculations. View full abstract»

29. Information transmission with additional noise
Publication Year: 1962 , Page(s): 293  304
Cited by: Papers (56)First Page of the ArticleView full abstract» 
30. On binary channels and their cascades
Publication Year: 1955 , Page(s): 19  27
Cited by: Papers (24)A detailed analysis of the general binary channel is given, with special reference to capacity (both separately and in cascade), input and output symbol distributions, and probability of error. The infinite number of binary channels with the same capacity lie on doublebranched equicapacity lines. Of the channels on the lower branch of a given equicapacity line, the symmetric channel has the smallest probability of error and the largest capacity in cascade, unless the capacity is small, in which case the asymmetric channel (with one noiseless symbol) has the smallest probability of error and the largest capacity in cascade. By simply reversing the designation of the output (or input) symbols, we can decrease the probability of error of any channel on the upper branch of the equicapacity line and increase the capacity in cascade of any asymmetric channel on the upper branch. In a binary channel neither symbol should be transmitted with a probability lying outside the interval if capacity is to be achieved. The maximally asymmetric input symbol distributions are approached by certain lowcapacity channels. For these channels, redundancy coding permits an appreciable fraction of capacity in cascade if sufficient delay can be tolerated. View full abstract»

31. On the Shannon theory of information transmission in the case of continuous signals
Publication Year: 1956 , Page(s): 102  108
Cited by: Papers (90)First Page of the ArticleView full abstract» 
32. A note on the maximum flow through a network
Publication Year: 1956 , Page(s): 117  119
Cited by: Papers (68)This note discusses the problem of maximizing the rate of flow from one terminal to another, through a network which consists of a number of branches, each of which has a limited capacity. The main result is a theorem: The maximum possible flow from left to right through a network is equal to the minimum value among all simple cutsets. This theorem is applied to solve a more general problem, in which a number of input nodes and a number of output nodes are used. View full abstract»

33. Tests on a cell assembly theory of the action of the brain, using a large digital computer
Publication Year: 1956 , Page(s): 80  93
Cited by: Papers (13)Theories by D.O. Hebb and P.M. Milner on how the brain works were tested by simulating neuron nets on the IBM Type 704 Electronic Calculator. The formation of cell assemblies from an unorganized net of neurons was demonstrated, as well as a plausible mechanism for shortterm memory and the phenomena of growth and fractionation of cell assemblies. The cell assemblies do not yet act just as the theory requires, but changes in the theory and the simulation offer promise for further experimentation. View full abstract»

34. Joint estimation of delay, Doppler, and Doppler rate
Publication Year: 1960 , Page(s): 330  341
Cited by: Papers (29)  Patents (1)The methods of inverse probability have been used by Woodward and others to obtain lower bounds on variances of radar parameter estimators. Previous results on the lower bounds of variances of delay and Doppler estimators have assumed that the reflecting object travels with a constant line of sight velocity and does not cause scintillation in the radar return. Using the inverse probability approach, this paper derives expressions for the minimum variances of estimators of the delay, Doppler, and Doppler rate of a radar return assumed to consist of a long train of pulses with independent scintillation from pulse to pulse. View full abstract»

35. On the factorization of rational matrices
Publication Year: 1961 , Page(s): 172  189
Cited by: Papers (146)Many problems in electrical engineering, such as the synthesis of linear n ports and the detection and filtration of multivariable systems corrupted by stationary additive noise, depend for their successful solution upon the factorization of a matrixvalued function of a complex variable . This paper presents several algorithms for affecting such decompositions for the class of rational matrices , i.e., matrices whose entries are ratios of polynomials in . The methods employed are elementary in nature and center around the Smith canonic form of a polynomial matrix. Several nontrivial examples are worked out in detail to illustrate the theory. View full abstract»

36. Spatial filtering in optics
Publication Year: 1956 , Page(s): 56  65
Cited by: Papers (18)  Patents (2)Starting with the formulation of H. H. Hopkins for the image forming properties of an optical system in terms of a coherence factor over the object plane, the two extreme cases of complete coherence and incoherence are considered. The incoherent case is treated briefly as a lowpass spatial frequency filter. In the case of coherent illumination, it is shown that the optical analog of such wellknown electrical concepts as equalization [17], edgesharpening, and the detection of periodic and isolated signals in the presence of noise can be carried out with relative ease. A detailed theoretical treatment of the problem together with illustrations emphasizes the analogy between optical and electrical filtering. View full abstract»

37. A probabilistic model for runlength coding of pictures
Publication Year: 1959 , Page(s): 157  163
Cited by: Papers (37)  Patents (3)A firstorder Markoff process representation for pictures is proposed in order to study the picture coding system known as runlength coding (differentialcoordinate encoding). A lower bound for the saving in channel capacity is calculated on the basis of this model, and is compared with the results obtained by previous investigators. In addition, this representation is shown to yield an insight into the runlength coding system which might not otherwise be obtained. The application of this probabilistic model to an "elastic" system of runlength coding is also discussed. View full abstract»

38. Processing gains against reverberation (clutter) using matched filters
Publication Year: 1960 , Page(s): 342  348
Cited by: Papers (25)The Woodward ambiguity function is discussed in connection with the output of a matched filter. A formula for the treatment of sonar reverberation or radar clutter is set up in terms of the ambiguity function. This formula is applied to determine the effect of signal waveform on the output signaltoreverberation power ratio of a matched filter for a simple distribution of randomized scatterers. View full abstract»

39. Predictive codingI
Publication Year: 1955 , Page(s): 16  24
Cited by: Papers (58)  Patents (3)Predictive coding is a procedure for transmitting messages which are sequences of magnitudes. In this coding method, the transmitter and the receiver store past message terms, and from them estimate the value of the next message term. The transmitter transmits, not the message term, but the difference between it and its predicted value. At the receiver this error term is added to the receiver prediction to reproduce the message term. This procedure is defined and messages, prediction, entropy, and ideal coding are discussed to provide a basis for Part II, which will give the mathematical criterion for the best predictor for use in the predictive coding of particular messages, will give examples of such messages, and will show that the error term which is transmitted in predictive coding may always be coded efficiently. View full abstract»

40. Some general aspects of the sampling theorem
Publication Year: 1956 , Page(s): 139  146
Cited by: Papers (27)The sampling theorem is recognized as an interpolation formula. Starting from the Lagrange Polynomial, this theorem is developed under conditions which are of broader applicability than those usually stated. Such a view point indicates the essential unity of temporal and frequency domain application. It will also be shown that the theorem is applicable as an exact interpolation formula throughout the complex plane. The basic theorem is extended to include sampling of the first derivative of the function. The concept of bandlimited functions is introduced through use of FourierStieltjes representations. This is then shown to be subsumed under the general class of functions which is considered in connection with the interpolation theorems developed. This approach, as presented, readily leads to the establishment of many sampling theorems. It is hoped that this paper will aid the formulation of particularly applicable sampling theorems for specific problems. View full abstract»

41. Handwriting and pattern recognition
Publication Year: 1962 , Page(s): 160  166
Cited by: Papers (27)Handwriting can be characterized as a sequence of basic strokes connected according to rule. Handwriting so generated approximates that of humans very closely. Such a matching process can be used as the fundamental principle in a handwriting recognizer. View full abstract»

42. Some properties of nonbinary errorcorrecting codes
Publication Year: 1958 , Page(s): 77  82
Cited by: Papers (38)An errorcorrecting code may be thought of as a subset of points belonging to a set in which a metric is defined such that the distance between every pair of distinct points of is larger than some given number. In Hamming's original formulation, was taken to be the set of all bit binary numbers and the distance between a pair of binary numbers and was taken to be the number of bits of which do not agree with the corresponding bits of . In this note we shall take to be the set of all tuples in which each coordinate of an tuple can assume one of integral values: , with . Properties of these nonbinary codes will be discussed. View full abstract»

43. Optical data processing and filtering systems
Publication Year: 1960 , Page(s): 386  400
Cited by: Papers (61)  Patents (6)Optical systems, which inherently possess two degrees of freedom rather than the single degree of freedom available in a single electronic channel, appear to offer some advantages over their electronic counterparts for certain applications. Coherent optical systems have the added property that one may easily obtain many successive twodimensional Fourier transforms of any given light amplitude distribution, or, by use of astigmatic optics, onedimensional transforms can be obtained. Therefore, most linear operations of an integral transform nature are easily implemented. The optical implementation of integral transforms which are of importance to communication theory is discussed; the general problems of optical filter synthesis and multichannel computation and data processing are introduced, followed by a discussion of potential applications. Astigmatic systems, which permit multichannel operation in lieu of twodimensional processing, are treated as a special case of general twodimensional processors. Complex input functions are discussed with relation to their role in coherent optical systems. View full abstract»

44. A note on learning signal detection
Publication Year: 1962 , Page(s): 126  128
Cited by: Papers (4)This paper considers a threshold detector operating on noisy binary pulses. It is shown that an adaptive errorcorrection procedure can bring the threshold to the point of equal false alarm and missed detection probabilities. View full abstract»

45. Leastsquare synthesis of radar ambiguity functions
Publication Year: 1962 , Page(s): 246  254
Cited by: Papers (66)The synthesis of radar ambiguity functions is approached by minimizing the integrated square error between an arbitrary desired function and a realizable ambiguity function. The approximation is carried out via an orthonormal signal basis which generates an "induced basis" over the timefrequency plane consisting of all pairwise crossambiguity functions of the signal basis. The minimum meansquare error and the corresponding signal are determined through an eigenvalue problem for 1) approximation by complex autoambiguity function and 2) approximation by complex crossambiguity function. A new form of the realizability theorem shows that the conditions for crossambiguity functions differ from those for autoambiguity functions only in the absence of the symmetry condition: . Moreover, the approximations by cross and autoambiguity functions coincide whenever the desired function has the above symmetry. When an ambiguity function realizable by a known signal is to be approximated on a finite basis, leastsquare approximation in signal space or in ambiguityfunction space leads to equivalent results. The relation between the meansquare errors in the two spaces is obtained. For the phase incoherent radar case an iteration procedure is presented for successively modifying the arbitrary phase assigned to the desired function. Convergence is proved in the sense that the error is nonincreasing at each stage of the iteration, but arrival at the best approximation to the desired magnitude is not guaranteed. As an aid in numerical applications, a formulation based on a discrete sample grid in the timefrequency plane is derived. With the appropriate grid dimensions, the analytic procedures carry over directly into sampleddata representation. View full abstract»

46. A note on the sampling principle for continuous signals
Publication Year: 1957 , Page(s): 143  146
Cited by: Papers (39)Two sampling (integral interpolation) theorems for continuous signals (continuous parameter stochastic processes) are proved. The first of these is the sampling principle introduced by Shannon, precise formulation or proof of which has not appeared hitherto. Obtained as a secondary result in this connection is a generalization of a result on the spectra of sampled signals given by Bennet. The second theorem is a stochastic version of the NewtonGauss interpolation formula as representative of a different class of sampling theorems. View full abstract»

47. A note on the sampling theorem
Publication Year: 1955 , Page(s): 47  48
Cited by: Papers (24)The human operator often perceives rate as well as amplitude information in sampling various displayed continuous parameters. It is therefore necessary to extend the Sampling Theorem to allow the analysis of certain manmachine relations. The result is stated and the required mathematics included in the Appendix. Certain distinct problem areas where this extension can be employed fruitfully are indicated. View full abstract»

48. Multierror correcting codes for a binary asymmetric channel
Publication Year: 1959 , Page(s): 71  78
Cited by: Papers (4)In an asymmetric binary channel, it may be sufficient to correct single Oerrors and detect double 0errors, for example, while correcting double 1errors and detecting quadruple 1errors. (A double 1error is said to occur when two of the l's of an input code character are delivered as O's at the output of the channel.) Minimum distance requirements are given for pairs of code characters of a code which corrects tuple 1errors, detects tuple 1errors, corrects tuple 0errors, and detects tuple Oerrors , and , are nonnegative integers with and ). These requirements are weaker than those for a symmetrical tuple error correcting, tuple error detecting, code and hence may be used to generally obtain more code characters for a given character length than are obtainable in the case. If the channel is highly asymmetric, it may be sufficient to detect and correct only one type of error. An earlier paper considered the case of single 1error correction and showed that it was always possible to obtain more code characters than exist in known single error correcting codes of equivalent character length except in cases where the symmetric code is "closepacked." In this paper codes are developed for tuple 1error correction which also yield more code characters than symmetrical tuple error correcting codes of the same length. The correction scheme is generally symbolcorrecting, but may require messagecorrection of binary sequences whose length is approximately that of the code characters. A double 1error correcting code is discussed in some detail and examples of code generation and correction are included. View full abstract»

49. A matched filter communication system for multipath channels
Publication Year: 1960 , Page(s): 367  373
Cited by: Papers (14)A matched filter communication system is described whose underlying principles are based on the Rake. The pointtopoint synchronous teletype system employs complex lumpedparameter networks to generate and receive a pair of longduration, broadband signals representing Mark and Space respectively. The receiver contains a pair of matched filters whose output is a narrow pulse when the matching waveform is applied. One advantage of the system, arising from the long duration of the signals, is an increase in energy per teletype baud when operating under a peak power limitation. Another is that multiple propagation paths due to ionospheric reflection are resolved by the broadband signals, resulting in the appearance of the multipath pattern at the output of the Mark or Space matched filter. Recombination of paths is achieved by means of a recirculating delay line tuned to the teletype baud rate in conjunction with parallel multiplierintegrators in the Mark and Space channels. The combination acts as a selfadjusting correlation detector for the multipath pattern. View full abstract»

50. Quaternary codes for pulsed radar
Publication Year: 1960 , Page(s): 400  408
Cited by: Papers (38)  Patents (1)A class of quaternary codes is described, and an algorithm for generating the codes is given. The codes have properties that make them useful for radar applications: 1) their autocorrelation consists of a single pulse, 2) their length can be any power of two, 3) each code can be paired with another code (its mate) of the same class in such a way that the crosscorrelation of mates is identically zero, 4) coded waveforms can be generated in a simple network the number of whose elements is proportional to the base2 logarithm of the code length, and 5) the same network can be readily converted to a matched filter for the coded waveform. View full abstract»
Aims & Scope
This Transactions ceased production in 1962. The current retitled publication is IEEE Transactions on Information Theory.
Further Links
Persistent Link: http://ieeexplore.ieee.org/servlet/opac?punumber=4547527 More »
Frequency: 6
ISSN:
00961000
Subjects
 Communication, Networking & Broadcasting
 Signal Processing & Analysis