The Emergent-Context Emergent-Input Framework for Temporal Processing

Many temporal processing tasks face great challenges. The handcrafted features and structure used in these temporal processing methods result in the brittle systems. To avoid handcrafted designs, we analyze and propose the EmergentContext Emergent-Input (ECEI) framework in this work. The Developmental Network (DN-1) can be considered as the first ECEI framework that used emergent motor vectors as temporal context (state) for top-down attention. This method suggests that the developmental root of temporal states is at the (open) motor end and always expressible by the motor end, not originally internal and hidden. Indeed, it has enabled the control of a Turing machine to incrementally emerge inside the network. However, DN-1 used a handcrafted partition of motor neurons as concept regions, which does not allow context to automatically emerge. The same is true for hidden (internal) areas other than the motor area. We analyze why this limitation is fundamentally damaging. To overcome this limitation, Developmental Network 2 (DN-2) is proposed with some new biology inspired mechanisms. DN2 is an ECEI framework which can directly use any naturally emerged motor patterns, allowing a freedom of thought that has been largely overlooked. The phoneme recognition experiments are used to investigate DN-2's performance with two different settings about the motor area: handcrafted motor concept zones and emergent motor area. Based on the experimental results, we compared and analyzed such two mechanisms. This work is a step toward our goal of empowering DN-2 to not only free thoughts, but such free-thoughts are also necessary for handling increasingly complicated temporal processing tasks.