By Topic

A Field Guide to Dynamical Recurrent Networks

Cover Image Copyright Year: 2001
Author(s): John F. Kolen; Stefan C. Kremer
Publisher: Wiley-IEEE Press
Content Type : Books & eBooks
Topics: Robotics & Control Systems
  • Print

Abstract

Acquire the tools for understanding new architectures and algorithms of dynamical recurrent networks (DRNs) from this valuable field guide, which documents recent forays into artificial intelligence, control theory, and connectionism. This unbiased introduction to DRNs and their application to time-series problems (such as classification and prediction) provides a comprehensive overview of the recent explosion of leading research in this prolific field. A Field Guide to Dynamical Recurrent Networks emphasizes the issues driving the development of this class of network structures. It provides a solid foundation in DRN systems theory and practice using consistent notation and terminology. Theoretical presentations are supplemented with applications ranging from cognitive modeling to financial forecasting. A Field Guide to Dynamical Recurrent Networks will enable engineers, research scientists, academics, and graduate students to apply DRNs to various real-world problems and learn about different areas of active research. It provides both state-of-the-art information and a road map to the future of cutting-edge dynamical recurrent networks.

  •   Click to expandTable of Contents

    • Full text access may be available. Click article title to sign in or learn about subscription options.

      Frontmatter

      Page(s): i - xxx
      Copyright Year: 2001

      Wiley-IEEE Press eBook Chapters

      The prelims comprise:
      Half Title
      IEEE Press Board Page
      Title
      Copyright
      Dedication
      Contents
      Preface
      Acknowledgments
      List of Figures
      List of Tables
      List of Contributors View full abstract»

    • Full text access may be available. Click article title to sign in or learn about subscription options.

      Introduction

      Page(s): 1
      Copyright Year: 2001

      Wiley-IEEE Press eBook Chapters

      Acquire the tools for understanding new architectures and algorithms of dynamical recurrent networks (DRNs) from this valuable field guide, which documents recent forays into artificial intelligence, control theory, and connectionism. This unbiased introduction to DRNs and their application to time-series problems (such as classification and prediction) provides a comprehensive overview of the recent explosion of leading research in this prolific field. A Field Guide to Dynamical Recurrent Networks emphasizes the issues driving the development of this class of network structures. It provides a solid foundation in DRN systems theory and practice using consistent notation and terminology. Theoretical presentations are supplemented with applications ranging from cognitive modeling to financial forecasting. A Field Guide to Dynamical Recurrent Networks will enable engineers, research scientists, academics, and graduate students to apply DRNs to various real-world problems and learn about different areas of active research. It provides both state-of-the-art information and a road map to the future of cutting-edge dynamical recurrent networks. View full abstract»

    • Full text access may be available. Click article title to sign in or learn about subscription options.

      Dynamical Recurrent Networks

      Page(s): 3 - 11
      Copyright Year: 2001

      Wiley-IEEE Press eBook Chapters

      This chapter contains sections titled:
      Introduction
      Dynamical Recurrent Networks
      Overview
      Conclusion View full abstract»

    • Full text access may be available. Click article title to sign in or learn about subscription options.

      Architectures

      Page(s): 13
      Copyright Year: 2001

      Wiley-IEEE Press eBook Chapters

      Acquire the tools for understanding new architectures and algorithms of dynamical recurrent networks (DRNs) from this valuable field guide, which documents recent forays into artificial intelligence, control theory, and connectionism. This unbiased introduction to DRNs and their application to time-series problems (such as classification and prediction) provides a comprehensive overview of the recent explosion of leading research in this prolific field. A Field Guide to Dynamical Recurrent Networks emphasizes the issues driving the development of this class of network structures. It provides a solid foundation in DRN systems theory and practice using consistent notation and terminology. Theoretical presentations are supplemented with applications ranging from cognitive modeling to financial forecasting. A Field Guide to Dynamical Recurrent Networks will enable engineers, research scientists, academics, and graduate students to apply DRNs to various real-world problems and learn about different areas of active research. It provides both state-of-the-art information and a road map to the future of cutting-edge dynamical recurrent networks. View full abstract»

    • Full text access may be available. Click article title to sign in or learn about subscription options.

      Networks with Adaptive State Transitions

      Page(s): 15 - 25
      Copyright Year: 2001

      Wiley-IEEE Press eBook Chapters

      This chapter contains sections titled:
      Introduction
      The Search for Context
      Recurrent Approaches to Context
      Representing Context
      Training
      Architectures
      Conclusion View full abstract»

    • Full text access may be available. Click article title to sign in or learn about subscription options.

      Delay Networks: Buffers to the Rescue

      Page(s): 27 - 38
      Copyright Year: 2001

      Wiley-IEEE Press eBook Chapters

      This chapter contains sections titled:
      Introduction to Delay Networks
      Back-Propagation Through Time Learning Algorithm
      Delay Networks With Feedback: NARX Networks
      Long-Term Dependencies in NARX Networks
      Experimental Results: The Latching Problem
      Conclusion View full abstract»

    • Full text access may be available. Click article title to sign in or learn about subscription options.

      Memory Kernels

      Page(s): 39 - 54
      Copyright Year: 2001

      Wiley-IEEE Press eBook Chapters

      This chapter contains sections titled:
      Introduction
      Different Types of Memory Kernels
      Generic Representation of a Memory Kernel
      Basis Issues
      Universal Approximation Theorem
      Training Algorithms
      Illustrative Example
      Conclusion View full abstract»

    • Full text access may be available. Click article title to sign in or learn about subscription options.

      Capabilities

      Page(s): 55
      Copyright Year: 2001

      Wiley-IEEE Press eBook Chapters

      Acquire the tools for understanding new architectures and algorithms of dynamical recurrent networks (DRNs) from this valuable field guide, which documents recent forays into artificial intelligence, control theory, and connectionism. This unbiased introduction to DRNs and their application to time-series problems (such as classification and prediction) provides a comprehensive overview of the recent explosion of leading research in this prolific field. A Field Guide to Dynamical Recurrent Networks emphasizes the issues driving the development of this class of network structures. It provides a solid foundation in DRN systems theory and practice using consistent notation and terminology. Theoretical presentations are supplemented with applications ranging from cognitive modeling to financial forecasting. A Field Guide to Dynamical Recurrent Networks will enable engineers, research scientists, academics, and graduate students to apply DRNs to various real-world problems and learn about different areas of active research. It provides both state-of-the-art information and a road map to the future of cutting-edge dynamical recurrent networks. View full abstract»

    • Full text access may be available. Click article title to sign in or learn about subscription options.

      Dynamical Systems and Iterated Function Systems

      Page(s): 57 - 81
      Copyright Year: 2001

      Wiley-IEEE Press eBook Chapters

      This chapter contains sections titled:
      Introduction
      Dynamical Systems
      Iterated Function Systems
      Symbolic Dynamics
      The DRN Connection
      Conclusion View full abstract»

    • Full text access may be available. Click article title to sign in or learn about subscription options.

      Representation of Discrete States

      Page(s): 83 - 102
      Copyright Year: 2001

      Wiley-IEEE Press eBook Chapters

      This chapter contains sections titled:
      Introduction
      Finite-State Automata
      Neural Network Representations of DFA
      Pushdown Automata
      Turing Machines
      Conclusion View full abstract»

    • Full text access may be available. Click article title to sign in or learn about subscription options.

      Simple Stable Encodings of FiniteState Machines in Dynamic Recurrent Networks

      Page(s): 103 - 127
      Copyright Year: 2001

      Wiley-IEEE Press eBook Chapters

      This chapter contains sections titled:
      Introduction
      Definitions
      Encoding
      Encoding of Mealy Machines in DRN
      Encoding of Moore Machines in DRN
      Encoding of Deterministic Finite-State Automata in DRN
      Conclusion
      Acknowledgments View full abstract»

    • Full text access may be available. Click article title to sign in or learn about subscription options.

      Representation beyond Finite States: Alternatives to Pushdown Automata

      Page(s): 129 - 142
      Copyright Year: 2001

      Wiley-IEEE Press eBook Chapters

      This chapter contains sections titled:
      Introduction
      Hierarchies of Languages and Machines
      DRNs and Nonregular Languages
      Generalization and Inductive Bias
      Conclusion View full abstract»

    • Full text access may be available. Click article title to sign in or learn about subscription options.

      Universal Computation and SuperTuring Capabilities

      Page(s): 143 - 151
      Copyright Year: 2001

      Wiley-IEEE Press eBook Chapters

      This chapter contains sections titled:
      Introduction
      The Model
      Preliminary: Computational Complexity
      Summary of Results
      Pondering Real Weights
      Analog Computation
      Conclusion
      Acknowledgments View full abstract»

    • Full text access may be available. Click article title to sign in or learn about subscription options.

      Algorithms

      Page(s): 153
      Copyright Year: 2001

      Wiley-IEEE Press eBook Chapters

      Acquire the tools for understanding new architectures and algorithms of dynamical recurrent networks (DRNs) from this valuable field guide, which documents recent forays into artificial intelligence, control theory, and connectionism. This unbiased introduction to DRNs and their application to time-series problems (such as classification and prediction) provides a comprehensive overview of the recent explosion of leading research in this prolific field. A Field Guide to Dynamical Recurrent Networks emphasizes the issues driving the development of this class of network structures. It provides a solid foundation in DRN systems theory and practice using consistent notation and terminology. Theoretical presentations are supplemented with applications ranging from cognitive modeling to financial forecasting. A Field Guide to Dynamical Recurrent Networks will enable engineers, research scientists, academics, and graduate students to apply DRNs to various real-world problems and learn about different areas of active research. It provides both state-of-the-art information and a road map to the future of cutting-edge dynamical recurrent networks. View full abstract»

    • Full text access may be available. Click article title to sign in or learn about subscription options.

      Insertion of Prior Knowledge

      Page(s): 155 - 177
      Copyright Year: 2001

      Wiley-IEEE Press eBook Chapters

      This chapter contains sections titled:
      Introduction
      Constrained Nondeterministic Insertion in First-Order Networks
      Second-Order Networks
      Other Related Techniques
      Conclusion View full abstract»

    • Full text access may be available. Click article title to sign in or learn about subscription options.

      Gradient Calculations for Dynamic Recurrent Neural Networks

      Page(s): 179 - 205
      Copyright Year: 2001

      Wiley-IEEE Press eBook Chapters

      This chapter contains sections titled:
      Introduction
      Learning in Networks with Fixed Points
      Computing the Gradient Without Assuming a Fixed Point
      Some Simulations
      Stability and Perturbation Experiments
      Other Non-Fixed-Point Techniques
      Learning with Scale Parameters
      Conclusion View full abstract»

    • Full text access may be available. Click article title to sign in or learn about subscription options.

      Understanding and Explaining DRN Behavior

      Page(s): 207 - 228
      Copyright Year: 2001

      Wiley-IEEE Press eBook Chapters

      This chapter contains sections titled:
      Introduction
      Performance Deterioration
      Dynamic Space Exploration
      DFA Extraction: Fool's Gold?
      Theoretical Foundations
      How Can DFA Outperform Networks?
      Alternative Extraction Methods
      Extension to Fuzzy Automata
      Application to Financial Forecasting
      Conclusion View full abstract»

    • Full text access may be available. Click article title to sign in or learn about subscription options.

      Limitations

      Page(s): 229
      Copyright Year: 2001

      Wiley-IEEE Press eBook Chapters

      Acquire the tools for understanding new architectures and algorithms of dynamical recurrent networks (DRNs) from this valuable field guide, which documents recent forays into artificial intelligence, control theory, and connectionism. This unbiased introduction to DRNs and their application to time-series problems (such as classification and prediction) provides a comprehensive overview of the recent explosion of leading research in this prolific field. A Field Guide to Dynamical Recurrent Networks emphasizes the issues driving the development of this class of network structures. It provides a solid foundation in DRN systems theory and practice using consistent notation and terminology. Theoretical presentations are supplemented with applications ranging from cognitive modeling to financial forecasting. A Field Guide to Dynamical Recurrent Networks will enable engineers, research scientists, academics, and graduate students to apply DRNs to various real-world problems and learn about different areas of active research. It provides both state-of-the-art information and a road map to the future of cutting-edge dynamical recurrent networks. View full abstract»

    • Full text access may be available. Click article title to sign in or learn about subscription options.

      Evaluating Benchmark Problems by Random Guessing

      Page(s): 231 - 235
      Copyright Year: 2001

      Wiley-IEEE Press eBook Chapters

      This chapter contains sections titled:
      Introduction
      Random Guessing (RG)
      Experiments
      Final Remarks
      Conclusion
      Acknowledgments View full abstract»

    • Full text access may be available. Click article title to sign in or learn about subscription options.

      Gradient Flow in Recurrent Nets: The Difficulty of Learning LongTerm Dependencies

      Page(s): 237 - 243
      Copyright Year: 2001

      Wiley-IEEE Press eBook Chapters

      This chapter contains sections titled:
      Introduction
      Exponential Error Decay
      Dilemma: Avoiding Aradient Decay Prevents Long-Term Latching
      Remedies
      Conclusion View full abstract»

    • Full text access may be available. Click article title to sign in or learn about subscription options.

      Limiting the Computational Power of Recurrent Neural Networks: VapnikChervonenkis Dimension and Noise

      Page(s): 245 - 254
      Copyright Year: 2001

      Wiley-IEEE Press eBook Chapters

      This chapter contains sections titled:
      Introduction
      Time-Bounded Networks and VC Dimension
      Robustness to Noise
      Conclusion
      Acknowledgments View full abstract»

    • Full text access may be available. Click article title to sign in or learn about subscription options.

      Applications

      Page(s): 255
      Copyright Year: 2001

      Wiley-IEEE Press eBook Chapters

      Acquire the tools for understanding new architectures and algorithms of dynamical recurrent networks (DRNs) from this valuable field guide, which documents recent forays into artificial intelligence, control theory, and connectionism. This unbiased introduction to DRNs and their application to time-series problems (such as classification and prediction) provides a comprehensive overview of the recent explosion of leading research in this prolific field. A Field Guide to Dynamical Recurrent Networks emphasizes the issues driving the development of this class of network structures. It provides a solid foundation in DRN systems theory and practice using consistent notation and terminology. Theoretical presentations are supplemented with applications ranging from cognitive modeling to financial forecasting. A Field Guide to Dynamical Recurrent Networks will enable engineers, research scientists, academics, and graduate students to apply DRNs to various real-world problems and learn about different areas of active research. It provides both state-of-the-art information and a road map to the future of cutting-edge dynamical recurrent networks. View full abstract»

    • Full text access may be available. Click article title to sign in or learn about subscription options.

      Dynamical Recurrent Networks in Control

      Page(s): 257 - 289
      Copyright Year: 2001

      Wiley-IEEE Press eBook Chapters

      This chapter contains sections titled:
      Introduction
      Description and Execution of TLRNN
      Elements of Training
      Basic Approach to Controller Synthesis
      Example 1
      Example 2
      Conclusion View full abstract»

    • Full text access may be available. Click article title to sign in or learn about subscription options.

      Sentence Processing and Linguistic Structure

      Page(s): 291 - 309
      Copyright Year: 2001

      Wiley-IEEE Press eBook Chapters

      This chapter contains sections titled:
      Introduction
      Case Studies: Dynamical Networks for Sentence Processing
      Conclusion View full abstract»

    • Full text access may be available. Click article title to sign in or learn about subscription options.

      Neural Network Architectures for the Modeling of Dynamic Systems

      Page(s): 311 - 350
      Copyright Year: 2001

      Wiley-IEEE Press eBook Chapters

      This chapter contains sections titled:
      Introduction and Overview
      Modeling Dynamic Systems by Feedforward Neural Networks
      Modeling Dynamic Systems by Recurrent Neural Networks
      Combining State-Space Reconstruction and Forecasting
      Conclusion View full abstract»

    • Full text access may be available. Click article title to sign in or learn about subscription options.

      From Sequences to Data Structures: Theory and Applications

      Page(s): 351 - 374
      Copyright Year: 2001

      Wiley-IEEE Press eBook Chapters

      This chapter contains sections titled:
      Introduction
      Historical Remarks
      Adaptive Processing of Structured Information
      Applications
      Conclusion View full abstract»

    • Full text access may be available. Click article title to sign in or learn about subscription options.

      Conclusion

      Page(s): 375
      Copyright Year: 2001

      Wiley-IEEE Press eBook Chapters

      Acquire the tools for understanding new architectures and algorithms of dynamical recurrent networks (DRNs) from this valuable field guide, which documents recent forays into artificial intelligence, control theory, and connectionism. This unbiased introduction to DRNs and their application to time-series problems (such as classification and prediction) provides a comprehensive overview of the recent explosion of leading research in this prolific field. A Field Guide to Dynamical Recurrent Networks emphasizes the issues driving the development of this class of network structures. It provides a solid foundation in DRN systems theory and practice using consistent notation and terminology. Theoretical presentations are supplemented with applications ranging from cognitive modeling to financial forecasting. A Field Guide to Dynamical Recurrent Networks will enable engineers, research scientists, academics, and graduate students to apply DRNs to various real-world problems and learn about different areas of active research. It provides both state-of-the-art information and a road map to the future of cutting-edge dynamical recurrent networks. View full abstract»

    • Full text access may be available. Click article title to sign in or learn about subscription options.

      Dynamical Recurrent Networks: Looking Back and Looking Forward

      Page(s): 377 - 378
      Copyright Year: 2001

      Wiley-IEEE Press eBook Chapters

      This chapter contains sections titled:
      Introduction
      The Challenges
      The Potential
      The Approaches
      The Successes
      Conclusion View full abstract»

    • Full text access may be available. Click article title to sign in or learn about subscription options.

      Bibliography

      Page(s): 379 - 408
      Copyright Year: 2001

      Wiley-IEEE Press eBook Chapters

      Acquire the tools for understanding new architectures and algorithms of dynamical recurrent networks (DRNs) from this valuable field guide, which documents recent forays into artificial intelligence, control theory, and connectionism. This unbiased introduction to DRNs and their application to time-series problems (such as classification and prediction) provides a comprehensive overview of the recent explosion of leading research in this prolific field. A Field Guide to Dynamical Recurrent Networks emphasizes the issues driving the development of this class of network structures. It provides a solid foundation in DRN systems theory and practice using consistent notation and terminology. Theoretical presentations are supplemented with applications ranging from cognitive modeling to financial forecasting. A Field Guide to Dynamical Recurrent Networks will enable engineers, research scientists, academics, and graduate students to apply DRNs to various real-world problems and learn about different areas of active research. It provides both state-of-the-art information and a road map to the future of cutting-edge dynamical recurrent networks. View full abstract»

    • Full text access may be available. Click article title to sign in or learn about subscription options.

      Glossary

      Page(s): 409 - 414
      Copyright Year: 2001

      Wiley-IEEE Press eBook Chapters

      Acquire the tools for understanding new architectures and algorithms of dynamical recurrent networks (DRNs) from this valuable field guide, which documents recent forays into artificial intelligence, control theory, and connectionism. This unbiased introduction to DRNs and their application to time-series problems (such as classification and prediction) provides a comprehensive overview of the recent explosion of leading research in this prolific field. A Field Guide to Dynamical Recurrent Networks emphasizes the issues driving the development of this class of network structures. It provides a solid foundation in DRN systems theory and practice using consistent notation and terminology. Theoretical presentations are supplemented with applications ranging from cognitive modeling to financial forecasting. A Field Guide to Dynamical Recurrent Networks will enable engineers, research scientists, academics, and graduate students to apply DRNs to various real-world problems and learn about different areas of active research. It provides both state-of-the-art information and a road map to the future of cutting-edge dynamical recurrent networks. View full abstract»

    • Full text access may be available. Click article title to sign in or learn about subscription options.

      Index

      Page(s): 415 - 422
      Copyright Year: 2001

      Wiley-IEEE Press eBook Chapters

      Acquire the tools for understanding new architectures and algorithms of dynamical recurrent networks (DRNs) from this valuable field guide, which documents recent forays into artificial intelligence, control theory, and connectionism. This unbiased introduction to DRNs and their application to time-series problems (such as classification and prediction) provides a comprehensive overview of the recent explosion of leading research in this prolific field. A Field Guide to Dynamical Recurrent Networks emphasizes the issues driving the development of this class of network structures. It provides a solid foundation in DRN systems theory and practice using consistent notation and terminology. Theoretical presentations are supplemented with applications ranging from cognitive modeling to financial forecasting. A Field Guide to Dynamical Recurrent Networks will enable engineers, research scientists, academics, and graduate students to apply DRNs to various real-world problems and learn about different areas of active research. It provides both state-of-the-art information and a road map to the future of cutting-edge dynamical recurrent networks. View full abstract»

    • Full text access may be available. Click article title to sign in or learn about subscription options.

      About the Editors

      Page(s): 423
      Copyright Year: 2001

      Wiley-IEEE Press eBook Chapters

      Acquire the tools for understanding new architectures and algorithms of dynamical recurrent networks (DRNs) from this valuable field guide, which documents recent forays into artificial intelligence, control theory, and connectionism. This unbiased introduction to DRNs and their application to time-series problems (such as classification and prediction) provides a comprehensive overview of the recent explosion of leading research in this prolific field. A Field Guide to Dynamical Recurrent Networks emphasizes the issues driving the development of this class of network structures. It provides a solid foundation in DRN systems theory and practice using consistent notation and terminology. Theoretical presentations are supplemented with applications ranging from cognitive modeling to financial forecasting. A Field Guide to Dynamical Recurrent Networks will enable engineers, research scientists, academics, and graduate students to apply DRNs to various real-world problems and learn about different areas of active research. It provides both state-of-the-art information and a road map to the future of cutting-edge dynamical recurrent networks. View full abstract»