Professional Documents
Culture Documents
Source Code
Writing the Backpropagation Algorithm into C++ Source Code
Feed forward networks are composed by neurons and layers. So, to make this porting
to source code easier, let's take the power of C++ classes and structures, and use them
to represent each portion of the neural network with them.
N EU RO N S TRU CTURE
The neuron structure should contain everything what a neuron represents:
neuron();//Constructor
~neuron();//Destructor
void create(int inputcount);//Allocates memory and initializates values
};
LA YER S TR UC TUR E
Our next structure is the layer. Basically, it contains an array of neurons along with the layer
input. All neurons from the layer share the same input, so the layer input is represented by an
array of floating point values.
struct layer
{
neuron **neurons;//The array of neurons
int neuroncount;//The total count of neurons
float *layerinput;//The layer input
int inputcount;//The total count of elements in layerinput
void create(int inputsize, int _neuroncount);//Creates the layer and allocates memory
void calculate();//Calculates all neurons performing the network formula
};
The layer structure contains a block of neurons representing a layer of the network. It contains
the pointer to array of neuron structure the array containing the input of the neuron and their
respective count descriptors. Moreover, it includes the constructor, destructor and creation
functions.
void propagate(const float *input);//Calculates the network values given an input pattern
//Updates the weight values of the network given a desired output and applying the backpropagation
//Algorithm
float train(const float *desiredoutput,const float *input,float alpha, float momentum);
//Returns the output layer..this is useful to get the output values of the network
inline layer &getOutput()
{
return m_outputlayer;
}
};
The bpnet class represents the entire neural network. It contains its basic input layer, output
layer and optional hidden layers.
Picturing the network structure it isnt that difficult. The trick comes when implementing the
training algorithm. Lets focus in the primary function bpnet::propagate(const float *input) and
the member function layer::calculate(); These functions what they do is to propagate and
calculate the neural network output values. Function propagate is the one you should use on your
final application.
FUNC T IO N
Our first goal is to calculate each layer neurons, and there is no better way than implementing a
member function in the layer object to do this job. Function layer::calculate() shows how to
void layer::calculate()
{
int i,j;
float sum;
//Apply the formula for each neuron
for(i=0;i<neuroncount;i++)
{
sum=0;//store the sum of all values here
for(j=0;j<inputcount;j++)
{
//Performing function
sum+=neurons[i]->weights[j] * layerinput[j]; //apply input * weight
}
sum+=neurons[i]->wgain * neurons[i]->gain; //apply the gain or theta multiplied by the gain weight.
//sigmoidal activation function
neurons[i]->output= 1.f/(1.f + exp(-sum));//calculate the sigmoid function
}
}
TR A IN ING T HE NE TW OR K
Finally, training the network is what makes the neural network useful. A neural network without
training does not really do anything. The training function is what applies the backpropagation
algorithm. I'll do my best to let you understand how this is ported to a program.
for(i=(m_hiddenlayercount - 1);i>=0;i--)
{
for(j=0;j<m_hiddenlayers[i]->neuroncount;j++)
{
output=m_hiddenlayers[i]->neurons[j]->output;
//calculate the error for this layer
errorc= output * (1-output) * sum;
//update neuron weights
for(k=0;k<m_hiddenlayers[i]->inputcount;k++)
{
delta=m_hiddenlayers[i]->neurons[j]->deltavalues[k];
udelta= alpha * errorc * m_hiddenlayers[i]->layerinput[k] + delta * momentum;
m_hiddenlayers[i]->neurons[j]->weights[k]+=udelta;
m_hiddenlayers[i]->neurons[j]->deltavalues[k]=udelta;
csum+=m_hiddenlayers[i]->neurons[j]->weights[k] * errorc;//needed for next layer
}
sum=csum;
csum=0;
}
for(j=0;j<m_inputlayer.inputcount;j++)
{
delta=m_inputlayer.neurons[i]->deltavalues[j];
udelta=alpha * errorc * m_inputlayer.layerinput[j] + delta * momentum;
//update weights
m_inputlayer.neurons[i]->weights[j]+=udelta;
m_inputlayer.neurons[i]->deltavalues[j]=udelta;
}
//and update the gain weight
m_inputlayer.neurons[i]->wgain+=alpha * errorc * m_inputlayer.neurons[i]->gain;
}
S AM P LE AP P LIC A T ION
The complete source code can be found at the end of this article. I also included a sample
application that shows how to use the class "bpnet" and how you may use it on an application.
The sample shows how to teach the neural network to learn the XOR (or exclusive) gate.
There isn't much complexity to create any application.
#include <iostream>
#include "bpnet.h"
using namespace std;
#define PATTERN_COUNT 4
#define PATTERN_SIZE 2
#define NETWORK_INPUTNEURONS 3
#define NETWORK_OUTPUT 1
#define HIDDEN_LAYERS 0
#define EPOCHS 20000
int main()
{
//Create some patterns
//playing with xor
//XOR input values
float pattern[PATTERN_COUNT][PATTERN_SIZE]=
{
{0,0},
{0,1},
{1,0},
{1,1}
};
net.create(PATTERN_SIZE,NETWORK_INPUTNEURONS,NETWORK_OUTPUT,HIDDEN_LAYERS,HIDDEN_LAYERS);
for(i=0;i<PATTERN_COUNT;i++)
{
net.propagate(pattern[i]);
//display result
cout << "TESTED PATTERN " << i << " DESIRED OUTPUT: " << *desiredout[i] << " NET RESULT: "<<
net.getOutput().neurons[0]->output << endl;
}
return 0;
}