Neural networks are a foundational concept in machine learning, inspired by the structure of the human brain. In this article, we will implement a basic neural network from scratch using Python and NumPy, without relying on deep learning libraries. Specifically, we will focus on two key operations:
- Forward pass: how input data flows through the network to generate predictions.
- Weight update: how the network learns by adjusting its parameters using gradient descent.
This tutorial assumes a basic understanding of vectors, dot products, and the sigmoid activation function.
1. Forward Pass
The forward pass is the process of passing input data through the network to produce output. It involves a series of matrix multiplications followed by activation functions applied at each layer.
We will implement a neural network with one hidden layer using sigmoid activation functions.
Code Implementation
import numpy as np # Sigmoid activation function def sigmoid(x): return 1 / (1 + np.exp(-x)) # Forward pass function def forward_pass(x, weights_input_to_hidden, weights_hidden_to_output): """ Perform a forward pass through a simple neural network. Parameters: - x: 1D numpy array representing the input features - weights_input_to_hidden: 2D numpy array (input layer to hidden layer weights) - weights_hidden_to_output: 2D numpy array (hidden layer to output layer weights) Returns: - hidden_layer_out: Output from the hidden layer after activation - output_layer_out: Final output from the network """ # Input to hidden layer hidden_layer_in = np.dot(x, weights_input_to_hidden) hidden_layer_out = sigmoid(hidden_layer_in) # Hidden to output layer output_layer_in = np.dot(hidden_layer_out, weights_hidden_to_output) output_layer_out = sigmoid(output_layer_in) return hidden_layer_out, output_layer_out
Example
# Input vector (1 sample, 3 features) x = np.array([0.5, -0.1, 0.2]) # Weights: input (3) to hidden (2) weights_input_to_hidden = np.array([ [0.1, -0.2], [0.4, 0.5], [-0.3, 0.2] ]) # Weights: hidden (2) to output (1) weights_hidden_to_output = np.array([ [0.3], [-0.1] ]) # Execute forward pass hidden_out, output_out = forward_pass(x, weights_input_to_hidden, weights_hidden_to_output) print("Hidden layer output:", hidden_out) print("Output layer output:", output_out)
This code processes a single input vector through the network and prints the hidden and output layer activations.
2. Updating Weights with Gradient Descent
After calculating the output from a forward pass, the next step in training is to update the weights. This is done using gradient descent, which adjusts weights to minimize the error between predicted and target values.
The function below performs a single epoch of weight updates for a simple neural network with one layer.
Code Implementation
def update_weights(weights, features, targets, learnrate): """ Perform a single epoch of gradient descent. Parameters: - weights: numpy array of model weights - features: pandas DataFrame of input features - targets: numpy array of target values - learnrate: learning rate for gradient descent Returns: - Updated weights after one epoch """ del_w = np.zeros(weights.shape) for x, y in zip(features.values, targets): # Forward pass output = sigmoid(np.dot(x, weights)) # Error calculation error = y - output # Gradient (error term) error_term = error * output * (1 - output) # Accumulate weight changes del_w += error_term * x # Apply average weight update weights += learnrate * del_w / features.shape[0] return weights
Example Usage
import pandas as pd # Input features (3 samples, 2 features each) features = pd.DataFrame([ [0.5, 1.5], [1.0, -1.0], [1.5, 0.5] ]) # Target values targets = np.array([1, 0, 1]) # Initial weights weights = np.array([0.1, -0.2]) # Learning rate learnrate = 0.5 # Perform weight update updated_weights = update_weights(weights, features, targets, learnrate) print("Updated weights:", updated_weights)
In this example, we perform a single training step (epoch) using batch gradient descent. The weights are updated based on the error from all training examples.
.png)
