Estimation of Information Efficiency for Digital Information Processing Devices

A wide class of digital data processing devices is presented - from smoothing filters to artificial neural networks using autoencoders as an example, the main purpose of which is to extract hidden information from input signals. The report proposes an approach to quantifying the information efficiency of information processing devices based on the concept of Kolmogorov Complexity, the dynamics of changes in entropy and the Kullback-Leibler divergence as a measure of the difference between probability distributions.