Electronic 'neural' net algorithm for maximum entropy solutions of ill-posed problems

An ill-posed problem does not provide sufficient information to obtain a unique solution. In such cases, a predetermined strategy must be used to choose a particular solution. A powerful and widely used technique is to select the solution with the maximum informational entropy. Here an algorithm suitable for solving ill-posed problems using the entropy of the solution as a regularizer is described. The algorithm has been simulated on a microcomputer as if implemented in a 'neural', i.e. multiply connected, net-type electronic circuit. Two types of problem are considered. First, where the constraints on the solution are hard, as in the loaded-dice problem, the maximum-entropy solution is shown to be achieved in the high gain limit of the net. Second, where the constraints are soft, as in the deconvolution of data corrupted by noise, a maximum-entropy solution is obtained directly. Prior knowledge of the solution is not required but can be introduced to the net so that the cross entropy is used as the regularizer. Finally, issues pertinent to the building of an actual circuit are discussed.<
>