Search All of the Math Forum:

Views expressed in these public forums are not endorsed by NCTM or The Math Forum.

Topic: ga optimization of nn weights
Replies: 5   Last Post: Feb 8, 2013 9:47 AM

 Messages: [ Previous | Next ]
 Greg Heath Posts: 6,319 Registered: 12/7/04
Re: ga optimization of nn weights
Posted: Feb 6, 2013 8:24 PM

....
> function mse_calc = mse_test(x, net, inputs, targets)
> % 'x' contains the weights and biases vector
> % in row vector form as passed to it by the
> % genetic algorithm. This must be transposed
> % when being set as the weights and biases
> % vector for the network.
> % To set the weights and biases vector to the
> % one given as input
> net = setwb(net, x);
> % To evaluate the ouputs based on the given
> % weights and biases vector
> y = net(inputs);
> % Calculating the mean squared error
> mse_calc = sum((y-targets).^2)/length(y);

No. This is only valid for vectors. For matrices
there are many equivalent forms

Neq = prod(size(targets))
e = targets - y;

MSE = sum(sum(e.^2))/Neq
MSE = sumsqr(e)/Neq
MSE = sse(e)/Neq
MSE = mse(e)

There are several ways to mitigate overfitting,

1. Reduce H so that Neq >>Nw. An adequate value of r = Neq/Nw depends on the data.
2. Adjust MSE above by replacing Neq with Ndof = Neq - Nw, the number of degrees of freedom for estimation.
3. Use a holdout (from training) validation subset to stop the minimization of MSEtrn when MSEval is a minimum.
4. Use regularization with the modified MSE

MSEreg = MSE + alpha*mse(x)

I don't remember how alpha is determined. Check the source codes of msereg and
trainbr; e.g.,

help msereg
doc msereg
type msereg

Hope this helps.

Greg

Date Subject Author
2/6/13 Greg Heath
2/6/13 Greg Heath
2/6/13 Greg Heath
2/6/13 Greg Heath
2/8/13 Greg Heath