Neural networks are commonly considered ‘universal function approximators’ and are theorized to be able to approximate any continuous function to an arbitrary degree of accuracy given a sufficiently complex structure.
The network connects a structured set of elements, characterized by hyperparameters and learned parameters:
| Element | Definition |
|---|---|
| Structure | |
| Neuron (or Node) | A single computational unit that receives inputs, processes them, and passes the result to the next layer. |
| Layers | A group of neurons operating together at a specific depth. They are organized into Input, Hidden, and Output layers. |
| Parameters | (Values learned by the network during training) |
| Weights | Numerical values that determine the strength of the connection between neurons. |
| Bias | A constant value added to the input of a neuron, allowing the activation function to be shifted for better data fitting. |
| Hyperparameters & Functions | (Values and choices set by the developer before training) |
| Network width | The number of neurons in each level. |
| Network depth | The number of layers in the network. |
| Activation Function | A mathematical function that introduces non-linearity, allowing the network to learn complex patterns. |
| Loss Function | The objective function that measures the error between the network’s prediction and the actual correct answer, optimized during training. |
try:
import executable_engineering as exe
except ImportError:
%pip install -q executable_engineering
import executable_engineering as exe
from executable_engineering import interactive_neural_networkInteractive Neural Network Training¶
The widget below demonstrates how width, depth, and optimization step size impact the network’s ability to approximate a target function (x)$.
Use the controls to adjust architecture (depth and width), input a custom target function, and either step manually or click Train to Convergence to automatically train the network until a target loss or plateau is met.
interactive_neural_network()Some more avenues to explore neural networks: