Search All of the Math Forum:

Views expressed in these public forums are not endorsed by NCTM or The Math Forum.

Notice: We are no longer accepting new posts, but the forums will continue to be readable.

Topic: Neural Network with Binary Inputs
Replies: 3   Last Post: Jan 31, 2013 9:40 AM

 Messages: [ Previous | Next ]
 Greg Heath Posts: 6,387 Registered: 12/7/04
Re: Neural Network with Binary Inputs
Posted: Jan 16, 2013 7:12 PM

"Jude" wrote in message <kd6vmo\$93u\$1@newscl01ah.mathworks.com>...
> I am using neural network tool box to prove a concept. I like to use binary inputs for my learning. Do we have any special learning algorithm available for binary inputs? (OR how should I modify this call (change any arguments for BI inputs?); ?newff(xll',y_learn,
>[20],{'tansig','tansig'},'trainbfg','learngdm','msereg');? to fit a binary inputs)

The only serious input recommendation I have is to use bipolar binary {-1 , 1} and 'tansig'
(default) for the hidden layer. In addition, why not tranpose xll once and for all instead of doing it in multiple commands?

For outputs: The transfer and learning functions depend on the type of target

Reals: 'purelin' and 'trainlm'(default)

Unipolar binary {0,1}: 'logsig' and 'trainscg'; %Use for classsification with vec2ind/ind2vec

Bipolar binary {-1,1}: 'tansig' and 'trainscg'

> I m using as follows:
> NETff = newff(xll',y_learn,[20],{'tansig','tansig'},'trainbfg','learngdm','msereg');

Why are you using validation stopping (default) AND 'msereg' ? Because H = 20 is
definitely overfitting? Just use a more practical value for H. See below.

What size are your input and target matrices?

For [I N ]and [O N] , you will have Neq = N*O equations, to estimate , Nw = (I+1)*H+(H+1)*O unknown weights. Without validation stopping or regularization, it is wise to keep Neq > r*Nw for r > 1, i.e.,

H < (Neq/r -O) / (I+O+1) % r >1

I have successfully used H small enough so that ~2 <= r <= ~ 8 to 20. For
smaller values I recommend val stopping or regularization. I feel better using this ratio as a guide rather than just using a very large value for H (like, um, 20?) and covering up by using both val stopping and reglarization.

> NETff.trainParam.epochs = 100000;

What is wrong with the default?

> NETff.trainParam.goal = 0.00001;

MSEgoal ~ 0.01*mean(var(ylearn')) % or (0.01) -> (0.005)

> NETff= train(NETff,xll',y_learn);

[ NETff tr Yff Eff ] = train(NETff,xll',y_learn);

> Yff = sim(NETff,xll');

Unnecessary

> Where xll? is a binary number, eg: 1010101010

Use bipolar binary

> Thanks.
> Jude

OKEY-DOKE

Greg

PS: try tr = tr and see all the goodies that are in that structure!

Date Subject Author
1/16/13 Greg Heath
1/31/13 Greg Heath