Neural Networks#
Neural networks are commonly considered ‘universal function approximators’ and are theorized to be able to approximate any continuous function to an arbitrary degree of accruacy given a sufficiently complex structure.
The network connects a structured set of elements, characterized by hyper parameters.
Element |
Definition |
|---|---|
Structure |
|
Neuron (or Node) |
A single computational unit that receives inputs, processes them, and passes the result to the next layer. |
Layers |
A group of neurons operating together at a specific depth. They are organized into Input, Hidden, and Output layers. |
Parameters |
(Values learned by the network during training) |
Weights |
Numerical values that determine the strength of the connection between neurons. |
Bias |
A constant value added to the input of a neuron, allowing the activation function to be shifted for better data fitting. |
Hyperparameters & Functions |
(Values and choices set by the developer before training) |
Network width |
the number of neurons in each level |
Network depth |
the number of layers in the network |
Activation Function |
A mathematical function that introduces non-linearity, allowing the network to learn complex patterns. |
Loss Function |
the objective function that measures the error between the network’s prediction and the actual correct answer which is optimized in the training process. |
!pip install Mint-NM
from Mint_NM import init_weights, tanh, tanh_derivative, forward, compute_loss, backward, plot_nn_diagram, init_model, forward_pass, backward_pass, step, reset_model, save_function, change_depth, change_width, draw_network, update_plots
import ipywidgets as widgets
import numpy as np
import matplotlib.pyplot as plt
import networkx as nx
from ipywidgets import VBox, HBox, Button, Text, Label, Output
from IPython.display import display, clear_output
output_plot = widgets.Output()
output_table = widgets.Output()
status_label = widgets.Label(value="Press 'Step' to begin training.")
function_input = Text(value='exp(-x/2)*sin(2*pi*x)', description='f(x):')
status_label = Label()
output_plot = Output()
network_plot = Output(layout={"height": "500px", "overflow": "auto"})
metrics_plot = Output()
save_button = Button(description='Save Function', button_style='success')
reset_button = Button(description='Reset Model', button_style='warning')
step_label = Label(value="Step:")
step_1 = Button(description='1')
step_10 = Button(description='10')
step_100 = Button(description='100')
step_1000 = Button(description='1000')
step_10000 = Button(description='10000')
depth_label = Label(value="Change Depth (layers):")
depth_add = Button(description='+1')
depth_sub = Button(description='-1')
width_label = Label(value="Change Width (neurons):")
width_add = Button(description='+1')
width_sub = Button(description='-1')
save_button.on_click(lambda b: save_function(function_input, output_plot, metrics_plot, network_plot, status_label))
reset_button.on_click(lambda b: reset_model(output_plot, metrics_plot, network_plot, status_label))
step_1.on_click(lambda b: step(1, output_plot, metrics_plot, network_plot))
step_10.on_click(lambda b: step(10, output_plot, metrics_plot, network_plot))
step_100.on_click(lambda b: step(100, output_plot, metrics_plot, network_plot))
step_1000.on_click(lambda b: step(1000, output_plot, metrics_plot, network_plot))
step_10000.on_click(lambda b: step(10000, output_plot, metrics_plot, network_plot))
depth_add.on_click(lambda b: change_depth(1, output_plot, metrics_plot, network_plot, status_label))
depth_sub.on_click(lambda b: change_depth(-1, output_plot, metrics_plot, network_plot, status_label))
width_add.on_click(lambda b: change_width(1, output_plot, metrics_plot, network_plot, status_label))
width_sub.on_click(lambda b: change_width(-1, output_plot, metrics_plot, network_plot, status_label))
top_controls = HBox([function_input, save_button, reset_button])
step_controls = HBox([step_label, step_1, step_10, step_100, step_1000, step_10000])
depth_controls = HBox([depth_label, depth_sub, depth_add])
width_controls = HBox([width_label, width_sub, width_add])
graph_row = HBox([output_plot, metrics_plot])
full_ui = VBox([top_controls, status_label, step_controls, depth_controls, width_controls, network_plot, graph_row])
display(full_ui)
init_model()
update_plots(output_plot, metrics_plot, network_plot)
Defaulting to user installation because normal site-packages is not writeable
Collecting Mint-NM
Downloading mint_nm-0.1.28-py3-none-any.whl.metadata (750 bytes)
Requirement already satisfied: numpy in /home/runner/.local/lib/python3.12/site-packages (from Mint-NM) (2.3.5)
Requirement already satisfied: matplotlib in /home/runner/.local/lib/python3.12/site-packages (from Mint-NM) (3.10.7)
Requirement already satisfied: networkx in /home/runner/.local/lib/python3.12/site-packages (from Mint-NM) (3.6)
Requirement already satisfied: ipywidgets in /home/runner/.local/lib/python3.12/site-packages (from Mint-NM) (8.1.8)
Requirement already satisfied: IPython in /home/runner/.local/lib/python3.12/site-packages (from Mint-NM) (9.8.0)
Requirement already satisfied: pyppeteer in /home/runner/.local/lib/python3.12/site-packages (from Mint-NM) (2.0.0)
Requirement already satisfied: nbconvert in /home/runner/.local/lib/python3.12/site-packages (from Mint-NM) (7.16.6)
Requirement already satisfied: scipy in /home/runner/.local/lib/python3.12/site-packages (from Mint-NM) (1.16.3)
Requirement already satisfied: plotly in /home/runner/.local/lib/python3.12/site-packages (from Mint-NM) (6.5.0)
Requirement already satisfied: decorator>=4.3.2 in /home/runner/.local/lib/python3.12/site-packages (from IPython->Mint-NM) (5.2.1)
Requirement already satisfied: ipython-pygments-lexers>=1.0.0 in /home/runner/.local/lib/python3.12/site-packages (from IPython->Mint-NM) (1.1.1)
Requirement already satisfied: jedi>=0.18.1 in /home/runner/.local/lib/python3.12/site-packages (from IPython->Mint-NM) (0.19.2)
Requirement already satisfied: matplotlib-inline>=0.1.5 in /home/runner/.local/lib/python3.12/site-packages (from IPython->Mint-NM) (0.2.1)
Requirement already satisfied: pexpect>4.3 in /usr/lib/python3/dist-packages (from IPython->Mint-NM) (4.9.0)
Requirement already satisfied: prompt_toolkit<3.1.0,>=3.0.41 in /home/runner/.local/lib/python3.12/site-packages (from IPython->Mint-NM) (3.0.52)
Requirement already satisfied: pygments>=2.11.0 in /usr/lib/python3/dist-packages (from IPython->Mint-NM) (2.17.2)
Requirement already satisfied: stack_data>=0.6.0 in /home/runner/.local/lib/python3.12/site-packages (from IPython->Mint-NM) (0.6.3)
Requirement already satisfied: traitlets>=5.13.0 in /home/runner/.local/lib/python3.12/site-packages (from IPython->Mint-NM) (5.14.3)
Requirement already satisfied: wcwidth in /home/runner/.local/lib/python3.12/site-packages (from prompt_toolkit<3.1.0,>=3.0.41->IPython->Mint-NM) (0.2.14)
Requirement already satisfied: parso<0.9.0,>=0.8.4 in /home/runner/.local/lib/python3.12/site-packages (from jedi>=0.18.1->IPython->Mint-NM) (0.8.5)
Requirement already satisfied: executing>=1.2.0 in /home/runner/.local/lib/python3.12/site-packages (from stack_data>=0.6.0->IPython->Mint-NM) (2.2.1)
Requirement already satisfied: asttokens>=2.1.0 in /home/runner/.local/lib/python3.12/site-packages (from stack_data>=0.6.0->IPython->Mint-NM) (3.0.1)
Requirement already satisfied: pure-eval in /home/runner/.local/lib/python3.12/site-packages (from stack_data>=0.6.0->IPython->Mint-NM) (0.2.3)
Requirement already satisfied: comm>=0.1.3 in /home/runner/.local/lib/python3.12/site-packages (from ipywidgets->Mint-NM) (0.2.3)
Requirement already satisfied: widgetsnbextension~=4.0.14 in /home/runner/.local/lib/python3.12/site-packages (from ipywidgets->Mint-NM) (4.0.15)
Requirement already satisfied: jupyterlab_widgets~=3.0.15 in /home/runner/.local/lib/python3.12/site-packages (from ipywidgets->Mint-NM) (3.0.16)
Requirement already satisfied: contourpy>=1.0.1 in /home/runner/.local/lib/python3.12/site-packages (from matplotlib->Mint-NM) (1.3.3)
Requirement already satisfied: cycler>=0.10 in /home/runner/.local/lib/python3.12/site-packages (from matplotlib->Mint-NM) (0.12.1)
Requirement already satisfied: fonttools>=4.22.0 in /home/runner/.local/lib/python3.12/site-packages (from matplotlib->Mint-NM) (4.61.0)
Requirement already satisfied: kiwisolver>=1.3.1 in /home/runner/.local/lib/python3.12/site-packages (from matplotlib->Mint-NM) (1.4.9)
Requirement already satisfied: packaging>=20.0 in /usr/lib/python3/dist-packages (from matplotlib->Mint-NM) (24.0)
Requirement already satisfied: pillow>=8 in /home/runner/.local/lib/python3.12/site-packages (from matplotlib->Mint-NM) (12.0.0)
Requirement already satisfied: pyparsing>=3 in /usr/lib/python3/dist-packages (from matplotlib->Mint-NM) (3.1.1)
Requirement already satisfied: python-dateutil>=2.7 in /usr/lib/python3/dist-packages (from matplotlib->Mint-NM) (2.8.2)
Requirement already satisfied: beautifulsoup4 in /home/runner/.local/lib/python3.12/site-packages (from nbconvert->Mint-NM) (4.14.3)
Requirement already satisfied: bleach!=5.0.0 in /home/runner/.local/lib/python3.12/site-packages (from bleach[css]!=5.0.0->nbconvert->Mint-NM) (6.3.0)
Requirement already satisfied: defusedxml in /home/runner/.local/lib/python3.12/site-packages (from nbconvert->Mint-NM) (0.7.1)
Requirement already satisfied: jinja2>=3.0 in /usr/lib/python3/dist-packages (from nbconvert->Mint-NM) (3.1.2)
Requirement already satisfied: jupyter-core>=4.7 in /home/runner/.local/lib/python3.12/site-packages (from nbconvert->Mint-NM) (5.9.1)
Requirement already satisfied: jupyterlab-pygments in /home/runner/.local/lib/python3.12/site-packages (from nbconvert->Mint-NM) (0.3.0)
Requirement already satisfied: markupsafe>=2.0 in /usr/lib/python3/dist-packages (from nbconvert->Mint-NM) (2.1.5)
Requirement already satisfied: mistune<4,>=2.0.3 in /home/runner/.local/lib/python3.12/site-packages (from nbconvert->Mint-NM) (3.1.4)
Requirement already satisfied: nbclient>=0.5.0 in /home/runner/.local/lib/python3.12/site-packages (from nbconvert->Mint-NM) (0.10.2)
Requirement already satisfied: nbformat>=5.7 in /home/runner/.local/lib/python3.12/site-packages (from nbconvert->Mint-NM) (5.10.4)
Requirement already satisfied: pandocfilters>=1.4.1 in /home/runner/.local/lib/python3.12/site-packages (from nbconvert->Mint-NM) (1.5.1)
Requirement already satisfied: webencodings in /home/runner/.local/lib/python3.12/site-packages (from bleach!=5.0.0->bleach[css]!=5.0.0->nbconvert->Mint-NM) (0.5.1)
Requirement already satisfied: tinycss2<1.5,>=1.1.0 in /home/runner/.local/lib/python3.12/site-packages (from bleach[css]!=5.0.0->nbconvert->Mint-NM) (1.4.0)
Requirement already satisfied: platformdirs>=2.5 in /usr/local/lib/python3.12/dist-packages (from jupyter-core>=4.7->nbconvert->Mint-NM) (4.5.0)
Requirement already satisfied: jupyter-client>=6.1.12 in /home/runner/.local/lib/python3.12/site-packages (from nbclient>=0.5.0->nbconvert->Mint-NM) (8.6.3)
Requirement already satisfied: pyzmq>=23.0 in /home/runner/.local/lib/python3.12/site-packages (from jupyter-client>=6.1.12->nbclient>=0.5.0->nbconvert->Mint-NM) (27.1.0)
Requirement already satisfied: tornado>=6.2 in /home/runner/.local/lib/python3.12/site-packages (from jupyter-client>=6.1.12->nbclient>=0.5.0->nbconvert->Mint-NM) (6.5.2)
Requirement already satisfied: fastjsonschema>=2.15 in /home/runner/.local/lib/python3.12/site-packages (from nbformat>=5.7->nbconvert->Mint-NM) (2.21.2)
Requirement already satisfied: jsonschema>=2.6 in /usr/lib/python3/dist-packages (from nbformat>=5.7->nbconvert->Mint-NM) (4.10.3)
Requirement already satisfied: attrs>=17.4.0 in /usr/lib/python3/dist-packages (from jsonschema>=2.6->nbformat>=5.7->nbconvert->Mint-NM) (23.2.0)
Requirement already satisfied: pyrsistent!=0.17.0,!=0.17.1,!=0.17.2,>=0.14.0 in /usr/lib/python3/dist-packages (from jsonschema>=2.6->nbformat>=5.7->nbconvert->Mint-NM) (0.20.0)
Requirement already satisfied: soupsieve>=1.6.1 in /home/runner/.local/lib/python3.12/site-packages (from beautifulsoup4->nbconvert->Mint-NM) (2.8)
Requirement already satisfied: typing-extensions>=4.0.0 in /usr/lib/python3/dist-packages (from beautifulsoup4->nbconvert->Mint-NM) (4.10.0)
Requirement already satisfied: narwhals>=1.15.1 in /home/runner/.local/lib/python3.12/site-packages (from plotly->Mint-NM) (2.13.0)
Requirement already satisfied: appdirs<2.0.0,>=1.4.3 in /home/runner/.local/lib/python3.12/site-packages (from pyppeteer->Mint-NM) (1.4.4)
Requirement already satisfied: certifi>=2023 in /usr/lib/python3/dist-packages (from pyppeteer->Mint-NM) (2023.11.17)
Requirement already satisfied: importlib-metadata>=1.4 in /home/runner/.local/lib/python3.12/site-packages (from pyppeteer->Mint-NM) (8.7.0)
Requirement already satisfied: pyee<12.0.0,>=11.0.0 in /home/runner/.local/lib/python3.12/site-packages (from pyppeteer->Mint-NM) (11.1.1)
Requirement already satisfied: tqdm<5.0.0,>=4.42.1 in /home/runner/.local/lib/python3.12/site-packages (from pyppeteer->Mint-NM) (4.67.1)
Requirement already satisfied: urllib3<2.0.0,>=1.25.8 in /home/runner/.local/lib/python3.12/site-packages (from pyppeteer->Mint-NM) (1.26.20)
Requirement already satisfied: websockets<11.0,>=10.0 in /home/runner/.local/lib/python3.12/site-packages (from pyppeteer->Mint-NM) (10.4)
Requirement already satisfied: zipp>=3.20 in /home/runner/.local/lib/python3.12/site-packages (from importlib-metadata>=1.4->pyppeteer->Mint-NM) (3.23.0)
Downloading mint_nm-0.1.28-py3-none-any.whl (9.4 kB)
Installing collected packages: Mint-NM
Successfully installed Mint-NM-0.1.28