7
\$\begingroup\$

I have implemented a working version of perceptron learning algorithm in C. Right now, it only works on single layer perceptrons and only takes two inputs. I plan on making it work with more than two inputs, but want to make sure I'm doing everything right first.

Here is the tutorial I used: https://www.spicelogic.com/Blog/Perceptron-Artificial-Neural-Networks-10

The training data is from a data file called "training_data.txt". In the data file:

  • first number in each line is first input
  • second number in each line is second input
  • third number in each line is the class.

The input is meant to be used by a camera on a vehicle to detect if an object is a pedestrian or another vehicle.

In the data file:

  • first number in each line is first input -- height to width ratio of object
  • second number in each line is second input -- how reflective the object is
  • third number in each line is the class -- 1 is a car, 2 is a person

Here are my files:

main.c:

/*************************************************************/
/* This is a program that implements the preceptron learning */
/* algorithm -- single layer. It gets the training set from  */
/* the file training_data.txt (change the macro to use a     */
/* different file). Right now it only works with two inputs. */
/*************************************************************/

/**********************************/
/* How the data file works:       */
/* ^Each line is a new set        */
/* ^the first number is input1    */
/* ^the second number is input2   */
/* ^the third number is the class */
/*     means the end of data      */
/**********************************/

#include <stdio.h>
#include <stdlib.h>
/* for rand() */
#include <time.h>
#include <unistd.h>
/* for sleep */
#include <stdbool.h>
/* for type bool */

#include "functions.h"
/* all function prototypes are here */

/* _______________ */
/*                 */
/*     MACROS      */
/* _______________ */
#define THRESHOLD .5
#define DATA_FILE "training_data.txt"
#define LEARNING_RATE -.2 
// NOTE: bias defined in functions.c

int main(void)
{
    /* ___________________ */
    /*                     */
    /*     FILE I/O        */
    /* ___________________ */

    /* open training_data.txt for reading */
    FILE * file_pointer = fopen(DATA_FILE, "r");

    /* make sure the file can be opened */
    if (file_pointer == NULL)
    {
        fprintf(stderr, "Cannot open training data file.\n");
        fprintf(stderr, "Check permissions for data file.\n");
        exit(1);
    }

    /* ___________________ */
    /*                     */
    /*     VARIABLES       */
    /* ___________________ */

    float input1 = 0, input2 = 0;   
    /* the inputs for the artifical neural network */

    float weight1 = 0, weight2 = 0;
    /* the weights for the artifical neural network */

    float threshold = 0;
    /* used in Activation Function */
    /* if summation of weighted inputs >= threshold,
     * Activation Function returns true. Otherwise,
     * Activation Function returns false. */


    /* get learning rate from macro */
    float learning_rate = LEARNING_RATE;

    float dot_product = 0;
    /* dot product = (a1 * b1) + (a2 * b2) + ... + (an * bn) */
    /* This will be the summation of all the weighted inputs.
     * This value will be given to the activation function. */

    /* What actual_output is the object being classified in? */
    int actual_output = 0;

    /* error = expected output - actual ouput */
    /* error is used in the update weight formula */
    float error = 0;

    /* I will keep track of if there are any incorrect
     * classifications left by using a boolean value.
     * true means there are still incorrect classifications
     * flase means all classifications are correct. */
    bool incorrectClassifications = true;

    printf("PERCEPTRON TRAINING ALGORITHM IMPLEMENTATION\n");

    /* ___________________ */
    /*                     */
    /*        INPUT        */
    /* ___________________ */

    /* We need to seed the random number generator. 
     * Otherwise, it will produce the same number every time
     * the program is run. */
    srand(time(NULL));
    /* I am using the current time to seed rand(). */

    /* get threshold from the user */
    //threshold = getThreshold();

    /* get threshold from macro */
    threshold = THRESHOLD;

    /* The weights will start off as random numbers in 
     * the range [0, 1]. */
    weight1 = ((float)rand()/(float)(RAND_MAX/1.0)); 
    weight2 = ((float)rand()/(float)(RAND_MAX/1.0)); 

    while (incorrectClassifications == true)
    {
        incorrectClassifications = false;

        /* Let's loop through all the data sets. */
        int i = 1;

        /* loop will break if input = 999 */
        while (1)
        {

            /* get input from the data set file */ 
            input1 = getInput(file_pointer);
            if (input1 == 999) break;
            input2 = getInput(file_pointer);

            /* ___________________ */
            /*                     */
            /*     CALCULATION     */
            /* ___________________ */

            /* sum the weighted inputs */
            dot_product = sumWeightedInputs(input1, input2, weight1, weight2);

            /* apply activation function to sum of weighted inputs */
            actual_output = activationFunction(dot_product, threshold); 

            /* ___________________ */
            /*                     */
            /*       OUPUT         */
            /* ___________________ */

            /* print which data set we are on */
            printf("Data Set %d\n", i);

            /* print the inputs */
            printf("\n"); // new line
            printf("Input 1 = %.2f\n", input1);
            printf("Input 2 = %.2f\n", input2);

            /* print the weights */
            printf("\n"); // new line
            printf("Weight 1 = %.2f\n", weight1);
            printf("Weight 2 = %.2f\n", weight2);

            /* print the summation of weighted inputs */
            printf("\n"); // new line
            printf("Summation = %.2f\n", dot_product);

            /* print the actual_output */
            printf("Object classified to class %d.\n", actual_output);

            /* check the output */
            error = checkOutput(file_pointer, actual_output);

            /* print the result */
            if (error == 0) printf("Ouput correct.\n");
            else
            {
                /* set incorrectClassifications to true 
                 * to loop through the data set once more */
                incorrectClassifications = true;

                printf("Output incorrect.\n");
                printf("Error = %.0f\n", error);

                /* we need to update the weights if *
                 * there is an error */
                weight1 = updateWeights(weight1, learning_rate, input1, error);
                weight2 = updateWeights(weight2, learning_rate, input2, error);

                /* print the new weights */
                printf("\n"); // new line;
                printf("NEW WEIGHTS: \n");
                printf("*** New weight 1: %.2f\n", weight1);
                printf("*** New weight 2: %.2f\n", weight2);
            } // ends else

            printf("\n");
            printf("-----------------------------------------------\n");
            printf("-----------------------------------------------\n\n");

            i++; //increment i

            sleep(1);

        } // ends while (1)

        /* set the file pointer back to beginning of file */
        rewind(file_pointer);

    } // ends while(incorrectClassifications == true)

    /* Print the final weights */
    printf("\n");
    printf("Final Weights: \n");
    printf("Weight 1: %.2f\n", weight1);    
    printf("Weight 2: %.2f\n", weight2);

    /* close the input file */
    fclose(file_pointer);

    return 0;

} // ends main()

functions.h:

#include <stdio.h>
/* for FILE type */

/* get inputs for the perceptron from a data file */
float getInput(FILE * ftp);

/* Ask the user for a threshold */
/* Note: for now I won't use this function, I'll just
 * use a macro for the threshold */
float getThreshold(void);

/* A function for the dot product (summation) of weighted
 * inputs (i.e., (input1 * weight1) + (input2 * weight2) ... */
float sumWeightedInputs(float input1, float input2, float weight1, float weight2);

/* Activation Function -- sees if the weights is greater 
 * than a certain number, called threshold, and returns 1
 * and 2 otherwise */
int activationFunction(float dot_product, float threshold);

/* Function checks the actual output of the perceptron's ouput 
 * against the training data set. */
float checkOutput(FILE * ftp, float actual_output);

/* a function to update weights */
float updateWeights(float weight, float learning_rate, float input, float error);

functions.c:

#include "functions.h"
#include <stdio.h>

/* I will use a macro to define bias */
#define BIAS 0

/*************/
/* GET INPUT */
/*************/

/* gets a input for the perceptron from a data file */
float getInput(FILE * ftp)
{
  float input;

  /* scan the input from the training data */
  fscanf(ftp, "%f ", &input);

  /* Return the input */
  return input;
}


/******************/
/* GET THRESHOLD  */
/******************/
/* Ask the user for a threshold */
/* Note: for now I won't use this function.
 * Instead of asking the user for the threshold,
 * I'll just define it as a macro. 
 */
float getThreshold(void)
{
  float threshold;

  printf("Threshold: ");
  scanf("%f", &threshold);

  return threshold;
}

/**********************/
/* SUM WEIGHTED INPUT */
/**********************/
/* A function for the dot product (summation) of weighted
 * inputs (i.e., (input1 * weight1) + (input2 * weight2) ... */
float sumWeightedInputs(float input1, float input2, float weight1, float weight2)
{
  /* sum means dot product here */
  float sum = 0;

  /* figure out the dot product here */
  sum = (input1 * weight1) + (input2 * weight2);
  sum = sum + BIAS; // add bias

  /* return sum */
  return sum;
}

/******************/
/* UPDATE WEIGHTS */
/******************/

/* This is the function that updates the weights 
 * if the neuron misclassified input. */
/* I am using this formual to update weights: 
 * new weight = old weight + (learning rate * current input * (error)
 * where error = expected output - actual output */
float updateWeights(float weight, float learning_rate, float input, float error)
{
    float new_weight = 0;

    /* use the formula here */
    new_weight = weight + (learning_rate * input * error);

    return new_weight;
}

/***********************/
/* ACTIVATION FUNCTION */
/***********************/

/* Activation Function -- sees if the weights is greater 
 * than a certain number, called threshold, and returns 1
 * and 0 therwise. */
int activationFunction(float dot_product, float threshold)
{
  if (dot_product >= threshold) return 1;
  /* object actual_outputified to class 1 */

  else return 2;
  /* object actual_outputified to class 2 */
}

/*****************/
/* CHECK OUTPUT  */
/*****************/

/* Function checks the actual output of the perceptron's ouput against the expected output. */
float checkOutput(FILE * ftp, float actual_output)
{
  float expected_output = 0;

  /* error = expected_ouput - actual_output */
  float error = 0;
  /* the value of error is needed in the update
   * weight formula */

  /* get expected output from data file */
  fscanf(ftp, "%f ", &expected_output);
  printf("\n"); // new line

  printf("Expected Output: %.2f\n", expected_output);
  printf("Actual Output: %.2f\n", actual_output);

  /* calculate error */
  error = expected_output - actual_output;
  return error;
}

training_data.txt:

1.1 .6 1 
1.2 .7 1
1.3 .75 1
1.4 .8 1
.1 .1 2
.2 .15 2
.3 .15 2
.4 .2 2
1.2 .4 1
.9 .4 1
.6 .2 2
1.5 .9 1
.3 .3 2
999

makefile:

all:
    clear
    gcc -Wall main.c functions.c -o perceptron

run:
    clear
    gcc -Wall main.c functions.c -o perceptron
    ./perceptron

Thank you anyone and everyone for any advice!

\$\endgroup\$
3
  • \$\begingroup\$ Here is the project on my GitHub if anyone wants to try it: github.com/benjaminsl/perceptron \$\endgroup\$
    – user91656
    Apr 10, 2017 at 22:52
  • \$\begingroup\$ is there anything in particular where you want advice? \$\endgroup\$
    – BenKoshy
    Apr 11, 2017 at 3:00
  • \$\begingroup\$ Main concerns: Am I implementing perceptron correctly? Should I use an array for inputs and weights if I want to allow more inputs? Is there a better way to represent the data in the data file? When should I change the learning rate? ... Other concerns: Am I commenting not enough, just right, or too much? Is the project architecture good? \$\endgroup\$
    – user91656
    Apr 11, 2017 at 3:27

1 Answer 1

2
\$\begingroup\$

Portability

With the exception of the call to sleep() this program compiles on the Windows 10 operating system using Visual Studio 2019, to improve the portability you might want to write your own implementation of sleep for non Linux systems. Before including unistd.h check to see if the code is compiling on Linux.

Include Guards

The header file functions.h is missing an include guard. Include guards prevent header files from being included multiple times. This is important because it can prevent compilation errors and prevent multiple definitions of constants. The most portable form of include guards are:

#ifndef CONSTANT
#define CONSTANT
// necessary definitions
#endif

Some C compilers also support #pragma once.

Write Self Document Code

One of the largest problems with successful software is that it needs to be maintained. A successful program will receive many features requests and some of those requests will be implemented. The maintenance may not be performed by the original developer. As part of this maintenance, comments and documentation will need to be maintained. Comments are important in code to explain why an algorithm is written the way it is, and generally that is the only reason to write comments. It is better to write self-documenting code as much as possible to reduce the amount of comments necessary.

Comments such as:

/* _______________ */
/*                 */
/*     MACROS      */
/* _______________ */
            /* ___________________ */
            /*                     */
            /*     CALCULATION     */
            /* ___________________ */

            /* sum the weighted inputs */

and

    /* ___________________ */
    /*                     */
    /*     FILE I/O        */
    /* ___________________ */

    /* open training_data.txt for reading */

waste space in the source file and will require unnecessary maintenance. The main() function is almost 200 lines of code and comments, which is generally more than 3 screens in most editors or IDEs, a general best practice in programming is that no function should be larger than a single screen because of reading comprehension, it is very difficult to understand functions larger than a single screen. Being larger than a single screen increases the risks of bugs in the code and decreases the maintainability of the code. I was able to reduce the size of main() by almost 100 lines by deleting unnecessary comments.

Code Flexibility

The program would be more flexible or usable if the user passed the name of the file in on the command line rather than having the name of the file hard coded. It might also be better to input the threshold and bias in the command line as well.

Declare the Variables as Needed

In the original version of C back in the 1970s and 1980s variables had to be declared at the top of the function. That is no longer the case, and a recommended programming practice to declare the variable as needed. In C the language doesn't provide a default initialization of the variable so variables should be initialized as part of the declaration. For readability and maintainability each variable should be declared and initialized on its own line.

\$\endgroup\$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.