Hyperdimensional Computing: An Algebra for Computing with Vectors

Hyperdimensional computing extends the traditional (von Neumann) model of computing with numbers, to computing with wide vectors, e.g. 10,000‐bit. Operations that correspond to the addition and multiplication of numbers, augmented by permutations of vector coordinates, allow us to build computers for tasks that are served poorly by today's computers. The hardware requirements are a unique match to 3D nanoscale circuit technology. The vector operations can be built into circuits and programmed in traditional ways. The encoding of information, however, is totally different and takes getting used to. Multiple items of information are encoded into a single vector and distributed over all vector components in a kind of holographic representation. Computing with holographically encoded vectors relies on the superabundance of approximately orthogonal vectors and on the properties of the operations, namely, that some are invertible, some distributive, and some distance‐preserving. Such properties are familiar to us from computing with numbers; now they form a foundation for computing with vectors. The goal of computing with vectors is to interpret and to act fast on rich sensory input. Sensory data and commands to actuators are coordinated in the high‐dimensional vector space, but raw sensory input must first be brought into the space. It is done with sensor‐specific preprocessors that can be designed by experts or trained as traditional neural nets and then frozen. Similarly, high‐dimensional vectors for actions are converted to commands for motors by actuator‐specific postprocessors.