Math Forum Discussions

Greg Heath

Posts: 5,696
Registered: 12/7/04

Re: ga optimization of nn weights
Posted: Feb 6, 2013 1:18 PM

Plain Text

"Greg Heath" <heath@alumni.brown.edu> wrote in message <keu1no$3ru$1@newscl01ah.mathworks.com>...
> % Subject: ga optimization of nn weights
> % From: Syed Umar Amin <syed.umar.amin@gmail.com>
> % Sent: Feb 4, 2013 11:17:22 AM
>
> getting error Index exceeds matrix dimensions.
> pls help

I don't have the GA Toolbox. However, this may help

close all, clear all, clc;
load cancer_dataset
whos
% Name Size Bytes Class Attributes
%
% cancerInputs 9x699 50328 double
% cancerTargets 2x699 11184 double

Inputs = cancerInputs;
Targets = cancerTargets;
[ I N ] = size(Inputs) % [ 9 699 ]
[O N ] = size(Targets) % [ 2 699 ]
minmaxin = minmax(Inputs(:)') % [ 0.1 1 ]
minmaxtar = minmax(Targets(:)') % [ 0 1 ]

% The 2 outputs are not independent since they sum to 1. If this
% constraint is enforced during training, by using only one output (e.g,
% using 'logsig' ), the number of independent training equations is
% N*O/2 . If the constraint is not enforced, it imay be N*O (I think!).

targets = Targets(1,:);
[ O N ] = size(targets)
Neq = N*O % 699 No. of independent equations
[ inputs, muin, stdin ] = zscore(Inputs')'; % Better for 'tansig' hidden activation

% For a feedforward MLP with an I-H-O (Input-Hidden-Output)
% node topology the number of unknown weights is
% Nw = (I+1)*H+(H+1)*O
% Hidden node upper bound (Neq > Nw)

Hub = -1+ceil((Neq-O)/(I+O+1)) % 63

% To mitigate noise and measurement error, try to choose H
% so that Neq >> Nw

H = 10 % MATLAB default (Also try smaller values)
net = patternnet(H); % For classification
net.layers{2}.transferFcn = 'logsig'; % For classification
net = configure(net, inputs, targets);
h = @(x) mse_test(x, net, inputs, targets);
ga_opts = gaoptimset('TolFun', 1e-2,'display','iter');
[x_ga_opt, err_ga] = ga(h, Nw, ga_opts);

function mse_calc = mse_test(x, net, inputs, targets)
% 'x' contains the weights and biases vector
% in row vector form as passed to it by the
% genetic algorithm. This must be transposed
% when being set as the weights and biases
% vector for the network.
%
% To set the weights and biases vector to the
% one given as input
net = setwb(net, x);
% To evaluate the ouputs based on the given
% weights and biases vector
y = net(inputs);
% Calculating the mean squared error
% >mse_calc = sum((y-targets).^2)/length(y);

% Better to normalize by the average target variance

mse_calc = sum((y-targets).^2)/mean(var(targets',1));
end

Since there is no overfitting mitigation using a regularization goal

help msereg
doc msereg

or validation stopping (using training set to minimize goal but
stopping when validation set error is minimized)

then you must try to minimize H provided validation errors are
acceptable.

Given the optimal value for H, you still should mitigate configure's
random choice of initial weights by designing 10 or more candidate
nets. Then choose the best candidate.

Then and only then predict the generalization error on unseen
data by using the test set (to be completely unbiased, it is only used
ONCE. If you want to get more candidates, you should repartition the
data trn/val/tst/ and design new nets).

You may wish to obtain summary statistics of the MSE using the
10 or more weight intialization trials for Hopt.

Hope this helps.

Greg