A Field Guide to Dynamical Recurrent Networks

Cover Image Copyright Year: 2001
Author(s): John F. Kolen; Stefan C. Kremer
Book Type: Wiley-IEEE Press
Content Type : Books
Topics: Robotics & Control Systems
  • Print

Abstract

Acquire the tools for understanding new architectures and algorithms of dynamical recurrent networks (DRNs) from this valuable field guide, which documents recent forays into artificial intelligence, control theory, and connectionism. This unbiased introduction to DRNs and their application to time-series problems (such as classification and prediction) provides a comprehensive overview of the recent explosion of leading research in this prolific field.

A Field Guide to Dynamical Recurrent Networks emphasizes the issues driving the development of this class of network structures. It provides a solid foundation in DRN systems theory and practice using consistent notation and terminology. Theoretical presentations are supplemented with applications ranging from cognitive modeling to financial forecasting.

A Field Guide to Dynamical Recurrent Networks will enable engineers, research scientists, academics, and graduate students to apply DRNs to various real-world pr blems and learn about different areas of active research. It provides both state-of-the-art information and a road map to the future of cutting-edge dynamical recurrent networks.

  •   Click to expandTable of Contents

  • Full text access may be available. Click article title to sign in or learn about subscription options.

    Architectures

    Copyright Year: 2001

    Wiley-IEEE Press eBook Chapters

  • Full text access may be available. Click article title to sign in or learn about subscription options.

    Networks with Adaptive State Transitions

    Copyright Year: 2001

    Wiley-IEEE Press eBook Chapters

    This chapter contains sections titled:

  • Introduction

  • The Search for Context

  • Recurrent Approaches to Context

  • Representing Context

  • Training

  • Architectures

  • Conclusion

  • View full abstract»

  • Full text access may be available. Click article title to sign in or learn about subscription options.

    Delay Networks: Buffers to the Rescue

    A Field Guide to Dynamical Recurrent Networks
    Copyright Year: 2001

    Wiley-IEEE Press eBook Chapters

    This chapter contains sections titled:

  • Introduction to Delay Networks

  • Back-Propagation Through Time Learning Algorithm

  • Delay Networks With Feedback: NARX Networks

  • Long-Term Dependencies in NARX Networks

  • Experimental Results: The Latching Problem

  • Conclusion

  • View full abstract»

  • Full text access may be available. Click article title to sign in or learn about subscription options.

    Memory Kernels

    A Field Guide to Dynamical Recurrent Networks
    Copyright Year: 2001

    Wiley-IEEE Press eBook Chapters

    This chapter contains sections titled:

  • Introduction

  • Different Types of Memory Kernels

  • Generic Representation of a Memory Kernel

  • Basis Issues

  • Universal Approximation Theorem

  • Training Algorithms

  • Illustrative Example

  • Conclusion

  • View full abstract»

  • Full text access may be available. Click article title to sign in or learn about subscription options.

    Capabilities

    A Field Guide to Dynamical Recurrent Networks
    Copyright Year: 2001

    Wiley-IEEE Press eBook Chapters

  • Full text access may be available. Click article title to sign in or learn about subscription options.

    Dynamical Systems and Iterated Function Systems

    A Field Guide to Dynamical Recurrent Networks
    Copyright Year: 2001

    Wiley-IEEE Press eBook Chapters

    This chapter contains sections titled:

  • Introduction

  • Dynamical Systems

  • Iterated Function Systems

  • Symbolic Dynamics

  • The DRN Connection

  • Conclusion

  • View full abstract»

  • Full text access may be available. Click article title to sign in or learn about subscription options.

    Representation of Discrete States

    A Field Guide to Dynamical Recurrent Networks
    Copyright Year: 2001

    Wiley-IEEE Press eBook Chapters

    This chapter contains sections titled:

  • Introduction

  • Finite-State Automata

  • Neural Network Representations of DFA

  • Pushdown Automata

  • Turing Machines

  • Conclusion

  • View full abstract»

  • Full text access may be available. Click article title to sign in or learn about subscription options.

    Simple Stable Encodings of FiniteState Machines in Dynamic Recurrent Networks

    A Field Guide to Dynamical Recurrent Networks
    Copyright Year: 2001

    Wiley-IEEE Press eBook Chapters

    This chapter contains sections titled:

  • Introduction

  • Definitions

  • Encoding

  • Encoding of Mealy Machines in DRN

  • Encoding of Moore Machines in DRN

  • Encoding of Deterministic Finite-State Automata in DRN

  • Conclusion

  • Acknowledgments

  • View full abstract»

  • Full text access may be available. Click article title to sign in or learn about subscription options.

    Representation beyond Finite States: Alternatives to Pushdown Automata

    A Field Guide to Dynamical Recurrent Networks
    Copyright Year: 2001

    Wiley-IEEE Press eBook Chapters

    This chapter contains sections titled:

  • Introduction

  • Hierarchies of Languages and Machines

  • DRNs and Nonregular Languages

  • Generalization and Inductive Bias

  • Conclusion

  • View full abstract»

  • Full text access may be available. Click article title to sign in or learn about subscription options.

    Universal Computation and SuperTuring Capabilities

    A Field Guide to Dynamical Recurrent Networks
    Copyright Year: 2001

    Wiley-IEEE Press eBook Chapters

    This chapter contains sections titled:

  • Introduction

  • The Model

  • Preliminary: Computational Complexity

  • Summary of Results

  • Pondering Real Weights

  • Analog Computation

  • Conclusion

  • Acknowledgments

  • View full abstract»

  • Full text access may be available. Click article title to sign in or learn about subscription options.

    Algorithms

    A Field Guide to Dynamical Recurrent Networks
    Copyright Year: 2001

    Wiley-IEEE Press eBook Chapters

  • Full text access may be available. Click article title to sign in or learn about subscription options.

    Insertion of Prior Knowledge

    A Field Guide to Dynamical Recurrent Networks
    Copyright Year: 2001

    Wiley-IEEE Press eBook Chapters

    This chapter contains sections titled:

  • Introduction

  • Constrained Nondeterministic Insertion in First-Order Networks

  • Second-Order Networks

  • Other Related Techniques

  • Conclusion

  • View full abstract»

  • Full text access may be available. Click article title to sign in or learn about subscription options.

    Gradient Calculations for Dynamic Recurrent Neural Networks

    A Field Guide to Dynamical Recurrent Networks
    Copyright Year: 2001

    Wiley-IEEE Press eBook Chapters

    This chapter contains sections titled:

  • Introduction

  • Learning in Networks with Fixed Points

  • Computing the Gradient Without Assuming a Fixed Point

  • Some Simulations

  • Stability and Perturbation Experiments

  • Other Non-Fixed-Point Techniques

  • Learning with Scale Parameters

  • Conclusion

  • View full abstract»

  • Full text access may be available. Click article title to sign in or learn about subscription options.

    Understanding and Explaining DRN Behavior

    A Field Guide to Dynamical Recurrent Networks
    Copyright Year: 2001

    Wiley-IEEE Press eBook Chapters

    This chapter contains sections titled:

  • Introduction

  • Performance Deterioration

  • Dynamic Space Exploration

  • DFA Extraction: Fool's Gold?

  • Theoretical Foundations

  • How Can DFA Outperform Networks?

  • Alternative Extraction Methods

  • Extension to Fuzzy Automata

  • Application to Financial Forecasting

  • Conclusion

  • View full abstract»

  • Full text access may be available. Click article title to sign in or learn about subscription options.

    Limitations

    A Field Guide to Dynamical Recurrent Networks
    Copyright Year: 2001

    Wiley-IEEE Press eBook Chapters

  • Full text access may be available. Click article title to sign in or learn about subscription options.

    Evaluating Benchmark Problems by Random Guessing

    A Field Guide to Dynamical Recurrent Networks
    Copyright Year: 2001

    Wiley-IEEE Press eBook Chapters

    This chapter contains sections titled:

  • Introduction

  • Random Guessing (RG)

  • Experiments

  • Final Remarks

  • Conclusion

  • Acknowledgments

  • View full abstract»

  • Full text access may be available. Click article title to sign in or learn about subscription options.

    Gradient Flow in Recurrent Nets: The Difficulty of Learning LongTerm Dependencies

    A Field Guide to Dynamical Recurrent Networks
    Copyright Year: 2001

    Wiley-IEEE Press eBook Chapters

    This chapter contains sections titled:

  • Introduction

  • Exponential Error Decay

  • Dilemma: Avoiding Aradient Decay Prevents Long-Term Latching

  • Remedies

  • Conclusion

  • View full abstract»

  • Full text access may be available. Click article title to sign in or learn about subscription options.

    Limiting the Computational Power of Recurrent Neural Networks: Vapnik-Chervonenkis Dimension and Noise

    A Field Guide to Dynamical Recurrent Networks
    Copyright Year: 2001

    Wiley-IEEE Press eBook Chapters

    This chapter contains sections titled:

  • Introduction

  • Time-Bounded Networks and VC Dimension

  • Robustness to Noise

  • Conclusion

  • Acknowledgments

  • View full abstract»

  • Full text access may be available. Click article title to sign in or learn about subscription options.

    Applications

    A Field Guide to Dynamical Recurrent Networks
    Copyright Year: 2001

    Wiley-IEEE Press eBook Chapters

  • Full text access may be available. Click article title to sign in or learn about subscription options.

    Dynamical Recurrent Networks in Control

    A Field Guide to Dynamical Recurrent Networks
    Copyright Year: 2001

    Wiley-IEEE Press eBook Chapters

    This chapter contains sections titled:

  • Introduction

  • Description and Execution of TLRNN

  • Elements of Training

  • Basic Approach to Controller Synthesis

  • Example 1

  • Example 2

  • Conclusion

  • View full abstract»

  • Full text access may be available. Click article title to sign in or learn about subscription options.

    Sentence Processing and Linguistic Structure

    A Field Guide to Dynamical Recurrent Networks
    Copyright Year: 2001

    Wiley-IEEE Press eBook Chapters

    This chapter contains sections titled:

  • Introduction

  • Case Studies: Dynamical Networks for Sentence Processing

  • Conclusion

  • View full abstract»

  • Full text access may be available. Click article title to sign in or learn about subscription options.

    Neural Network Architectures for the Modeling of Dynamic Systems

    A Field Guide to Dynamical Recurrent Networks
    Copyright Year: 2001

    Wiley-IEEE Press eBook Chapters

    This chapter contains sections titled:

  • Introduction and Overview

  • Modeling Dynamic Systems by Feedforward Neural Networks

  • Modeling Dynamic Systems by Recurrent Neural Networks

  • Combining State-Space Reconstruction and Forecasting

  • Conclusion

  • View full abstract»

  • Full text access may be available. Click article title to sign in or learn about subscription options.

    From Sequences to Data Structures: Theory and Applications

    A Field Guide to Dynamical Recurrent Networks
    Copyright Year: 2001

    Wiley-IEEE Press eBook Chapters

    This chapter contains sections titled:

  • Introduction

  • Historical Remarks

  • Adaptive Processing of Structured Information

  • Applications

  • Conclusion

  • View full abstract»

  • Full text access may be available. Click article title to sign in or learn about subscription options.

    Conclusion

    A Field Guide to Dynamical Recurrent Networks
    Copyright Year: 2001

    Wiley-IEEE Press eBook Chapters

  • Full text access may be available. Click article title to sign in or learn about subscription options.

    Dynamical Recurrent Networks: Looking Back and Looking Forward

    A Field Guide to Dynamical Recurrent Networks
    Copyright Year: 2001

    Wiley-IEEE Press eBook Chapters

    This chapter contains sections titled:

  • Introduction

  • The Challenges

  • The Potential

  • The Approaches

  • The Successes

  • Conclusion

  • View full abstract»

  • Full text access may be available. Click article title to sign in or learn about subscription options.

    Bibliography

    A Field Guide to Dynamical Recurrent Networks
    Page(s): 379 - 408
    Copyright Year: 2001

    Wiley-IEEE Press eBook Chapters

    Acquire the tools for understanding new architectures and algorithms of dynamical recurrent networks (DRNs) from this valuable field guide, which documents recent forays into artificial intelligence, control theory, and connectionism. This unbiased introduction to DRNs and their application to time-series problems (such as classification and prediction) provides a comprehensive overview of the recent explosion of leading research in this prolific field.

    A Field Guide to Dynamical Recurrent Networks emphasizes the issues driving the development of this class of network structures. It provides a solid foundation in DRN systems theory and practice using consistent notation and terminology. Theoretical presentations are supplemented with applications ranging from cognitive modeling to financial forecasting.

    A Field Guide to Dynamical Recurrent Networks will enable engineers, research scientists, academics, and graduate students to apply DRNs to various real-world pr blems and learn about different areas of active research. It provides both state-of-the-art information and a road map to the future of cutting-edge dynamical recurrent networks. View full abstract»

  • Full text access may be available. Click article title to sign in or learn about subscription options.

    Glossary

    A Field Guide to Dynamical Recurrent Networks
    Page(s): 409 - 414
    Copyright Year: 2001

    Wiley-IEEE Press eBook Chapters

    Acquire the tools for understanding new architectures and algorithms of dynamical recurrent networks (DRNs) from this valuable field guide, which documents recent forays into artificial intelligence, control theory, and connectionism. This unbiased introduction to DRNs and their application to time-series problems (such as classification and prediction) provides a comprehensive overview of the recent explosion of leading research in this prolific field.

    A Field Guide to Dynamical Recurrent Networks emphasizes the issues driving the development of this class of network structures. It provides a solid foundation in DRN systems theory and practice using consistent notation and terminology. Theoretical presentations are supplemented with applications ranging from cognitive modeling to financial forecasting.

    A Field Guide to Dynamical Recurrent Networks will enable engineers, research scientists, academics, and graduate students to apply DRNs to various real-world pr blems and learn about different areas of active research. It provides both state-of-the-art information and a road map to the future of cutting-edge dynamical recurrent networks. View full abstract»

  • Full text access may be available. Click article title to sign in or learn about subscription options.

    Index

    A Field Guide to Dynamical Recurrent Networks
    Page(s): 415 - 422
    Copyright Year: 2001

    Wiley-IEEE Press eBook Chapters

    Acquire the tools for understanding new architectures and algorithms of dynamical recurrent networks (DRNs) from this valuable field guide, which documents recent forays into artificial intelligence, control theory, and connectionism. This unbiased introduction to DRNs and their application to time-series problems (such as classification and prediction) provides a comprehensive overview of the recent explosion of leading research in this prolific field.

    A Field Guide to Dynamical Recurrent Networks emphasizes the issues driving the development of this class of network structures. It provides a solid foundation in DRN systems theory and practice using consistent notation and terminology. Theoretical presentations are supplemented with applications ranging from cognitive modeling to financial forecasting.

    A Field Guide to Dynamical Recurrent Networks will enable engineers, research scientists, academics, and graduate students to apply DRNs to various real-world pr blems and learn about different areas of active research. It provides both state-of-the-art information and a road map to the future of cutting-edge dynamical recurrent networks. View full abstract»

  • Full text access may be available. Click article title to sign in or learn about subscription options.

    About the Editors

    A Field Guide to Dynamical Recurrent Networks
    Copyright Year: 2001

    Wiley-IEEE Press eBook Chapters



  • On This Page

    Recently Published

    Learn More About