Problems of performance measurement in locally-organized systems

We show that many of the modern intelligent systems belong to a wide class of distributed systems. The external behavior of such a system is governed with criteria which induce only partial ordering among various systems. This partial order does not allow building an analytical metrics in the space of such systems, the fact making systems largely indistinguishable. We explain the existence of many versions of AI systems made for the same purpose, like expert systems shells, which are frequently differentiated only through their secondary properties. The situation may be compared with that of the use of Pareto sets in theory of games where all different solutions belonging to a Pareto set are considered to be intrinsically similar.