Building a Basic Neural Network

Summary

This chapter explains neural network functionality using a basic neural network that readers can build from scratch using Python. A neural network relies on numerous computation units, the neurons, arranged into hierarchical layers. Each neuron accepts inputs from all its predecessors and provides outputs to its successors until the neural network as a whole satisfies a requirement. The first building block of the network is the activation function. The activation function determines which value is high or low enough to reflect a decision point in the neural network for a particular neuron or group of neurons. An optimizer serves to ensure that readers' neural network performs fast and correctly models whatever problem they want to solve by modifying the neural network's biases and weights. Each optimizer has completely different parameters to tune. The learning rate can affect both the time the neural network takes to learn a good solution.