Information-Theoretical Aspects of Inductive and Deductive Inference

By a straightforward application of Bayes' theorem of probability, the behavior is discussed of the credibilities (inductive probabilities) of competing hypotheses as functions of an increasing body of relevant empirical data. It is shown how the effect of a priori credibilities persists in the evaluation of credibilities in general, except in the important limiting cases investigated. An “inverse H-theorem” is mathematically demonstrated, according to which the entropy function defined in terms of the credibilities shows a net decrease in time. This decrease is not necessarily monotonous in an individual case, but is monotonous in the “expected” behavior of the inductive entropy function. Three machine-simulation experiments of inductive inference on the IBM 704 are described. The first two concern the classical problem of guessing the ratio of white and black balls in an urn. The third experiment concerns guessing a hidden pattern obeyed by a sequence of binary numbers.