By Topic

Information Theory, Transactions of the IRE Professional Group on

Issue 3 • Date March 1954

Filter Results

Displaying Results 1 - 13 of 13
  • Passage of stationary processes through linear and non-linear devices

    Publication Year: 1954 , Page(s): 4 - 25
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1260 KB)  

    Many problems in the theory of noise and other random functions can be formulated as the problem of finding the probability distribution of the functional $u = intlimits^infty_0 K(t') , V, (X (t'))dt'$ where K(t) and V(x) are known functions and x(t) is a random function of known statistical properties. The problem of finding the probability distribution of the noise output of a receiver consisting of a filter, a detector, and a second filter is of this type. Methods will be discussed which have led to solutions of this problem in some special cases. In the case of multidimensionally Markoffian x(t) the problem will be shown to be equivalent to an integral equation, which in many cases of interest reduces to a differential equation. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Statistical theory of signal detection

    Publication Year: 1954 , Page(s): 26 - 51
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (2134 KB)  

    A complete theory of detection is presented, which is capable of treating general types of signals (e.g. periodic, aperiodic, random) in noise of arbitrary statistical character. By proper formulation of the detection problem as a test of statistical hypotheses, the precise structure of the optimum detector can be specified and minimum detectable signals uniquely determined. For threshold reception (the problem of main interest) two classes of operation arise: if detection is coherent, as far as dependence on the input signal-to-noise ratio is concerned one has a linear system, no matter how weak the signal; on the other hand for incoherent reception one always has a quadratic dependence on this input ratio (modulation suppression). Threshold reception in these two instances requires respectively a suitably weighted cross-correlation of the received data with the a priori known signal, or a suitably weighted autocorrelation of the received data with itself. The optimum detector is in general a computer, involving non-linear operations and terminating in a decision operation, which depends on the type of statistical test (e.g. Neyman-Pearson, Ideal, Sequential, etc.) defining the observer. The threshold of decision is necessarily determined by a suitable betting or cost curve. (Both discrete (digital) and continuous (analog) sampling of the data are considered.) In this way optimum performance, consistent with the external constraints, is specified, and the extent by which actual systems depart from this limiting optimum can be calculated. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • The detectability of random signals in the presence of noise

    Publication Year: 1954 , Page(s): 52 - 62
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (769 KB)  

    This morning you have heard excellent presentations of two fields of endeavor, the results and techniques of which could be basic to a statistical theory of communication engineering. On the one hand, the field of statistical inference, as applied to discrete stochastic processes, has developed to a refined point due to the efforts of many statisticians. ï¿¿ï¿¿ï¿¿ The work of the late Professor Wald in his successful application of Von Neumann's game theory to the construction of a general theory of decision functions has played a dominant role in the development of these refinements. On the other hand, the theory of stochastic processes depending upon a discrete or continuous time parameter has been developed during the last three deoades by various mathematicians, only during the last few years has the study of statistical inference problems for continuous stochastic processes received much attention. Here the outstanding contribution is the thesis of Ulf Grenander, published in the Arkiv fï¿¿ï¿¿r matematik, Band 1, Hï¿¿ï¿¿fte 3, 1950. In attempting to apply the techniques of statistical inference to continuous processes, it is evident that the central problem is to obtain a coordinate system for the process which allows one to actually carry out. the computations called for by various statistical methods. As far as I am aware, there are at present only two types of continuous stochastic processes for which a coordinate system has been obtained with which one can carry through some of the computations necessary in the testing of statistical hypotheses. One process is a projection on to the real axis of a finite dimensional Markoff process, Gaussian or non Gaussian. The other process is a Gaussian process with a continuous covariance function. The restriction of a continuous covariance function is not serious, since this property applies to all of the stochastic models which have been set up to study continuous processes occurring in communication engineering. (The assumption - hat the spectrum of a process is a "pure white noise" is not consistent with continuity of the covariance function, but a pure white noise is merely a mathematical idealization. The process with a flat band limited spectrum ï¿¿ï¿¿ï¿¿ a model often used in application ï¿¿ï¿¿ï¿¿ does possess a continuous covariance function.) On the other hand, the restriction to Gaussian processes is one which would be desirable to remove in some cases. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • The response of linear systems to non-Gaussian noise

    Publication Year: 1954 , Page(s): 63 - 67
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (464 KB)  

    A fairly broad class of problems deals with the way certain properties of noise are altered on passage through a linear system. Fig. l defines these properties. If any n instants of time are chosen and if boundary value problems of the zero crossing type are omitted, the specification of the n dimensional probability distribution yields the most complete statistical information. Very often, however, this information is difficult or impossible to find and it is useful to obtain properties (2) and (3) of Fig. 1 without actually knowing property (1). Briefly, an nth order random process is defined by no more than an n dimensional probability distribution. Given a higher order distribution, it can be reduced to order n. The example in Fig. 1 shows how, for a 2nd degree (a Markoff) process, a trivariate form can be expressed in terms of bivariate and lower forms. A stationary process is one whose statistical properties do not depend on the choice of time reference. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Estimation of signal parameters in the presence of noise

    Publication Year: 1954 , Page(s): 68 - 89
    Cited by:  Papers (4)
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1288 KB)  

    This paper is concerned with certain applications of the estimation theory of Fisher and Cramer[1] to the problem of estimating signal parameters in the presence of noise. Specifically, the situation to be treated is as follows. A received signal View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • The use of the method of maximum likelihood in estimating continuous-modulated intelligence which has been corrupted by noise

    Publication Year: 1954 , Page(s): 90 - 105
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (950 KB)  

    A signal is received in the time interval (t ¿ T ¿ ¿ ¿ t). It is known that this signal is composed of noise plus intelligence a(t) which is statistical in nature and which has been modulated in some known way. Assuming that both intelligence and noise are Gaussian (although not necessarily stationary) time series, the analog of the classical maximum-likelihood estimate for ¿(t) is derived. The advantage of this approach is that it can handle arbitrary types of modulation. For unmodulated stationary intelligence and stationary noise, the solution reduces to that of Zadeh and Ragazzini. In the general case, the optimum estimate is given as the solution of a pair of integral equations. The amplitude-modulated case is treated in some detail. The application of the maximum-likelihood technique to problems involving arbitrary modulation was first suggested, as far as the author is aware, by F. W. Lehan and R. J. Parks of the Jet Propulsion Laboratory. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Detection of modulated noise-like signals

    Publication Year: 1954 , Page(s): 106 - 122
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (835 KB)  

    The detection of several interfering modulated noise-like signals is described and an expression is derived for the resultant correlation function, The correlation function is used to demonstrate the modulation suppression action of a linear detector for the general case of signals of noise-like character. Since the detection process suppresses the temporal variations of the weaker signals in one portion of the output spectrum and the stronger signals in another, it is possible, under certain conditions, to effectively separate the modulation information of the original signals. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Statistically almost optimum nonlinear network design

    Publication Year: 1954 , Page(s): 123
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (283 KB)  

    The statistically-optimum unstable filter to separate desired signals from noise and other undesired signals is shown to take the form of a high gain amplifier, with several output channels, each one of which computes the best statistical measure of one of the signals or the noise. The sum of these outputs comprises a negative feedback circuit. The imposition of the requirement of stability on the overall system is shown to be the same as the imposition of stability on each component plus a small correction in the zero locations. The addition of nonlinear components follows from differences in the amplitude probability distributions. The methods of automatically compensating for the unalterable dynamic characteristics of the output device, of the high gain amplifier, and for changes in the ratio of signal to noise power, are shown. The information needed for this design is the power density spectrum or autocorrelation function of each signal and noise, and the amplitude probability distribution of each. The mechanics of the design are quite simple, and can be done with only a conventional knowledge of circuit theory. A new philosophy of statistically optimum systems is formulated, which embraces both linear and nonlinear systems, and eliminates the necessity of minimizing the error power. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Simpie games of strategy occurring in communication through natural languages

    Publication Year: 1954 , Page(s): 124 - 137
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (1327 KB)  

    The purpose of the investigation to be reported here differs in one essential respect from the purpose of most statistical studies on problems of communication. We wish to state this difference from the beginning, in order to avoid misunderstanding of the methods used later on. Most communication problems are, broadly speaking, engineering problems; that is, they deal with the construction and evaluation of new methods of communication, using one's knowledge of "Nature," acquired elsewhere. Our problem is, broadly speaking, physical; we seek to improve our knowledge, or at least our description, of "Nature" by using models in which some statistical concepts, developed in the study of recent communication problems, are used side by side with such concepts of physics as were already used to solve older engineering problems. This difference of purpose will also explain an unessential character of this paper: the calculations are much simpler than in other papers of the symposium, because here one could choose the problems leading to simple theories by the present methods (but not by previous ones), whereas in other papers the problems were imposed. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Application of linear graphs to communication problems

    Publication Year: 1954 , Page(s): 138
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (246 KB)  

    Codes have been discussed recently in connection with signal compression and noise reduction in communication theory, and they are here defined as transformations between two time series of discrete symbols. The electronic apparatus which performs the transformation, or any part of the apparatus, can be described by a linear graph which represents transitions between different memory-states. It will be shown how these graphs can be used to systematically classify codes, combine different blocks of apparatus, examine synchronism of decoding apparatus, etc. Linear graphs have been studied as mathematical systems in connection with topology, Markov chains, and electrical network theory; hence there is already much theory which might be profitably applied to communication problems. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Mlnimum-cost encoding of information

    Publication Year: 1954 , Page(s): 139 - 149
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (921 KB)  

    Haying determined a cost in energy, time, or money of transmitting each of a set of symbols (e.g., dot and dash, pulses of various amplitudes, etc.), one may inquire as to the nature of a code using these symbols which will transmit a given amount of information at the least cost, or will transmit information at a given rate for the least cost per unit time. Such a code must use each symbol with a relative frequency given by a negative exponential of a linear combination of its cost and duration in the case of a noiseless communication channel. If noise is present, the cost of each symbol is effectively increased by a multiple of its "prevarication," i.e., the entropy of the received symbol when only the given symbol is transmitted. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Generalized servomechanism evaluation

    Publication Year: 1954 , Page(s): 150
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (256 KB)  

    In the course of staking general evaluations of servo systems utilizing magnetic amplifiers, the inadequacy of classical methods becomes immediately evident. Statistical treatment of the signal and the system error is neccessary criteria of performance other than the quadratic are essential; and it is possible that accurate evaluation can result only from including the effects of the inherent non-linearities of the system. Progress to date has resulted in three separate approaches to the problem: a) A computer which includes a system analog, and an error criterion and integrating circuit, and which accepts statistically typical signals from a signal storage or generating device. There results a single number describing the performance of the system. b) An analytical method applicable only to linear systems, which makes use of the atrtocorrelation function of the signal and noise inputs, and also of the higher moment correlation fkutions of the two. c) An analytical method applicable in instanrces when the signal spectrum changesabruptly. The anaysis is applicable only to linear systems and considers a quadratic criterion. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.
  • Optimum pulse-time determination

    Publication Year: 1954 , Page(s): 151 - 159
    Save to Project icon | Request Permissions | Click to expandQuick Abstract | PDF file iconPDF (689 KB)  

    A common problem in radar, navigation systems or pulse-time modulation systems is that of determination of the delay or time of arrival of a pulse or other distinctive modulation form. while considerable effort has been directed toward the optimization in various senses of the signal-to-noise ratios for such signals, the problem of the most accurate determination of time delay of signals in noise bas, with a few noteworthy exceptions, been taken for granted. It is of interest to examine this problem for its own sake, not only as indicative of optimum circuit practice, but, more important, in order to make clear the fundamental accuracy limitations and the manner in which these limitations depend upon signal and noise parameters. The consideration of this paper is limited to the case where there is no doubt as to the existence nor approximate position of a pulse but where the accuracy of determination of pulse position is limited by noise. This case is therefore more representative of the problems of high accuracy systems such as Loran or shoran than of radar. View full abstract»

    Full text access may be available. Click article title to sign in or learn about subscription options.

Aims & Scope

This Transactions ceased production in 1954. The current retitled publication is IEEE Transactions on Information Theory.

Full Aims & Scope