Neural networks in signal and image processing

Neural network (NN) research has gone a long way in the past decade. NNs now consist of many thousands of highly interconnected processing elements that can encode, store and recall relationships between different patterns by altering the weighting coefficients of inputs in a systematic way. Although they can generate reasonable outputs from unknown input patterns, and they can tolerate a great deal of noise, they are very slow when run on a serial machine. There also exists a combinatorial relationship between the number of neurons or processing elements (PEs) and the number of connections in the network. Therefore, in simulating a NN on a serial machine, one has to take into consideration the two different types of data available, namely connection data and output data, that each requires a large memory to be stored on. Some problems connected to this include floating point arithmetic for continuous values, as well as overheads required to tie all these data together and to exchange information between processors. In this paper we review the ALOPEX algorithms developed by us, discuss their complexities and give some examples of their applications in biomedical engineering problems. We do not claim that we know how the brain really works nor that we are able to build a computer that emulates the brain. NNs provide a theory of how information is stored in memory but not what is put in there.