Search All of the Math Forum:
Views expressed in these public forums are not endorsed by
Drexel University or The Math Forum.
|
|
|
|
Re: Neural Network with Binary Inputs
Posted:
Jan 31, 2013 9:40 AM
|
|
IT IS CONSIDERED A HEINOUS CRIME TO TOP-POST.
PLEASE REFRAIN FROM WRITING REPLIES ABOVE PREVIOUS ENTRIES.
i HAVE REMOVED YOUR REPLY TO THE END.
"Jude" wrote in message <kebrnc$cve$1@newscl01ah.mathworks.com>...
> "Greg Heath" <heath@alumni.brown.edu> wrote in message <kd7fkr$2h7$1@newscl01ah.mathworks.com>...
> > "Jude" wrote in message <kd6vmo$93u$1@newscl01ah.mathworks.com>...
> > > I am using neural network tool box to prove a concept. I like to use binary inputs for my learning. Do we have any special learning algorithm available for binary inputs? (OR how should I modify this call (change any arguments for BI inputs?); ?newff(xll',y_learn,
> > >[20],{'tansig','tansig'},'trainbfg','learngdm','msereg');? to fit a binary inputs)
> >
> > The only serious input recommendation I have is to use bipolar binary {-1 , 1} and 'tansig'
> > (default) for the hidden layer. In addition, why not tranpose xll once and for all instead of doing it in multiple commands?
> >
> > For outputs: The transfer and learning functions depend on the type of target
> >
> > Reals: 'purelin' and 'trainlm'(default)
> >
> > Unipolar binary {0,1}: 'logsig' and 'trainscg'; %Use for classsification with vec2ind/ind2vec
> >
> > Bipolar binary {-1,1}: 'tansig' and 'trainscg'
> >
> > > I m using as follows:
> > > NETff = newff(xll',y_learn,[20],{'tansig','tansig'},'trainbfg','learngdm','msereg');
> >
> > Why are you using validation stopping (default) AND 'msereg' ? Because H = 20 is
> > definitely overfitting? Just use a more practical value for H. See below.
> >
> How can I change, so validation will not stop?
> NETff = newff(xll',y_learn,[20],{'tansig','tansig'},'trainbfg','learngdm');
>
>
> > What size are your input and target matrices?
> >
> > For [I N ]and [O N] , you will have Neq = N*O equations, to estimate , Nw = (I+1)*H+(H+1)*O unknown weights. Without validation stopping or regularization, it is wise to keep Neq > r*Nw for r > 1, i.e.,
> >
> > H < (Neq/r -O) / (I+O+1) % r >1
> >
> > I have successfully used H small enough so that ~2 <= r <= ~ 8 to 20. For
> > smaller values I recommend val stopping or regularization. I feel better using this ratio as a guide rather than just using a very large value for H (like, um, 20?) and covering up by using both val stopping and reglarization.
> >
> > > NETff.trainParam.epochs = 100000;
> >
> > What is wrong with the default?
> >
> > > NETff.trainParam.goal = 0.00001;
> >
> > MSEgoal ~ 0.01*mean(var(ylearn')) % or (0.01) -> (0.005)
> >
> > > NETff= train(NETff,xll',y_learn);
> >
> > [ NETff tr Yff Eff ] = train(NETff,xll',y_learn);
> >
> > > Yff = sim(NETff,xll');
> >
> > Unnecessary
> >
> > > Where xll? is a binary number, eg: 1010101010
> >
> > Use bipolar binary
> >
> > > Thanks.
> > > Jude
> >
> > OKEY-DOKE
> >
> > Greg
> >
> > PS: try tr = tr and see all the goodies that are in that structure!
> Hi Greg,
>
> Thank you for your help. I incorporated most of your inputs.
>
> My issues are:
> Once I complete the NN training on the GUI (nntraintool) it shows 10 inputs (I 've given 10 binary inputs. However when I type my net on the command prompt (NETff) it only shows one input.
>
> NETff =
> dimensions:
>
> numInputs: 1
> numLayers: 3
> numOutputs: 1
> Is matlab automatically change my binary inputs to scalar?
I think you mean vector inputs to scalar.
No. The one input is 10-dimensional.
However, your binary inputs are changed to real values for computational purposes.
Hope this helps.
Greg
P.S. Unless you have a specific reason for doing otherwise, only use one hidden layer.
One hidden layer is sufficent for a universal approximator.
|
|
|
|
