Artificial Neural Network Classifier in Matlab

Question

I am trying to build a neural network classifier. I have created a neural network with 1 hidden layer (25 neurons) and 1 output layer (1 neuron/binary classification).

The dataset I am using has the following dimensions:

size(X_Train): 125973 x 122
size(Y_Train): 125973 x 1
size(X_Test): 22543 x 122
size(Y_test): 22543 x 1

My overall goal is to compare different training functions. But, I would like first to get your feedback about my code and how I improve it.

% Neural Network Binary-classification

clear ; close all; clc

%% =========== Part 1: Loading Data =============

%% Load Training Data
fprintf('Loading Data ...\n');

load('dataset.mat'); % training data stored in arrays X, y
X_training=X_training';
Y_training=Y_training';
X_testing=X_testing';
Y_testing=Y_testing';

%% Create the neural network
% 1, 2: ONE input, TWO layers (one hidden layer and one output layer)
% [1; 1]: both 1st and 2nd layer have a bias node
% [1; 0]: the input is a source for the 1st layer
% [0 0; 1 0]: the 1st layer is a source for the 2nd layer
% [0 1]: the 2nd layer is a source for your output
net = network(1, 2, [1; 1], [1; 0], [0 0; 1 0], [0 1]);
net.inputs{1}.size = 122; % input size
net.layers{1}.size = 25; % hidden layer size
net.layers{2}.size = 1; % output layer size

%% Transfer function in layers
net.layers{1}.transferFcn = 'logsig';
net.layers{2}.transferFcn = 'logsig';

net.layers{1}.initFcn = 'initnw';
net.layers{2}.initFcn = 'initnw';

net=init(net);

%% divide data into training and test
net.divideFcn= 'dividerand';
net.divideParam.trainRatio = 60/100; % 80% training
net.divideParam.valRatio = 20/100; % 20% validation set
net.divideParam.testRatio = 20/100; % 20% validation set

net.performFcn = 'crossentropy';

%% Training functions
net.trainFcn = 'trainscg'; %Scaled conjugate gradient backpropagation

%% Train the neural network
[net,tr] = train(net,X_training,Y_training); % return the network and training record

%% Test the Neural Network on the training set
outputs = net(X_training);
errors = gsubtract(Y_training,outputs);
performance = perform(net,Y_training,outputs);

%% Plots  (%training)
figure, plotperform(tr)
figure, plottrainstate(tr)

%% Test the Neural Network on the testing test
outputs1 = net(X_testing);
errors1 = gsubtract(Y_testing,outputs1);
performance1 = perform(net,Y_testing,outputs1);

figure, plotconfusion(Y_testing,outputs1)
figure, ploterrhist(errors1)

Below if the validation curve.

Confusion Matrix (Training set)

Confusion Matrix (Testing set)

Any remarks?

Edit:

I have used feature scaling or normalization:

net.performParam.normalization = 'standard';

which has improved the overall accuracy:

For more information, I have added the error histogram:

Answer 1

I'm not a code reviewer, but I'd like to comment on the design of your network, which you certainly seem to be doing just fine.

• It might be rather difficult to make any judgment, given that the application is undefined, while it seems you are designing a neural-network based detector.

• Numerically speaking, you might focus on your validation performance, by constantly redesigning your network architecture (e.g., number of hidden layers, number of hidden neurons, reducing and increasing batch sizes, training functions/methods as you mentioned, etc.), input preprocessing (e.g., smoothing, input interpolation or extrapolation in case possible, artifact removal, etc.).

• Not knowing what your datasets might be and how sophisticated that may be, you may focus on 10^-3 to 10^-6 convergence range. It might increase the performance (e.g., confusion matrices) of your network.

Overall, it seems your input is pretty stochastic. Input preprocessing may be worth looking into.