"Sriram " <firstname.lastname@example.org> wrote in message <email@example.com>... > I used the neural network toolbox ( nprtool ) for classifying my detected objects into either of 3 classes. I used 14 parameters (image moments) for all the 3 classes of input for training. As of now, I was able to collect only few data for each classes say around > > Class 1 - 17 * 14 > > Class 2 - 11 * 14 > >Class 3 - 48 * 14 > > Total Inputclass - 14*76 && Outputclass - 3 *76 > > I arranged these data in column wise to feed into nprtool box.... I used the default toolbox functions like - hidden as 10 , training - 70% (54 samples) , validation - 15% (11 samples), testing - 15% (11 samples) and started the training. > > Total number of Iterations was - 20 > >Performance (MSE) was - 0.0731 > > Gradient was 0.0617 > > Validation checks was - 6 > > Once the net has been created, I tried to use some data in "sim(net,input)" to > >check my networks performance. For certain inputs from the trained data set, the > >network's >performance was fine but for many it was very bad. (unexpected results).
Neither nonquantitative terms like "fine" and "bad" nor the quantitative result MSE = 0.0731 helps much without references. With target matrix
a reasonable reference is MSE00 = mean(var(t',1)) = 0.17671 so that the normalized MSE is NMSE = MSE/MSE00 = 0.41368 and the resulting R^2 statistic ( see "coefficient-of-determination in Wikipedia) is R2 = 1-MSE = 0.58632 which is not necessarily good.
Even with a reference MSE, a classifier should be graded on error rate; not MSE.
What is needed is NMSE and PCTerr for each class (1-3) and each data split (trn/val/tst)
> >This is my status and problem. Now I need suggestions - > > In what all ways I can improve the performance of the network.
1. Duplicate copies of class1 and class2 so that you have 48 vectors of each class (N = 3*48) = 142 2. Standardize the I = 14 input variables to zero-mean/unit-variance (zscore or mapstd) 3. The columns of the target matrix should be columns of the O=3-dimensional unit matrix eye(3). For example, t = repmat( eye(3), 1, 48 ). The corresponding classindices are obtained from classind = vec2ind(t); 4. For a 0.7/0.15/0.15 data split Ntrn = N-round(0.15*N) = 100 resulting in Ntrneq = Ntrn*O = 300 training equations 5. For an I-H-O network topology, there will be Nw =(I+1)*H+(H+1)*O = 3 + 18*H unknown weights to estimate with Ntrneq equations. This results in Ndof = Ntrneq-Nw estimation degrees of freedom. It is desirable that Ntrneq >> Nw or H << 297/18 = 16.5. However, larger values of H can be tolerated by using validation stopping or regularization using trainbr. 6. The normalizing factor for the MSE should be the average variance of the target matrix variables
MSE00 = mean(var(t',1)) % Biased (divided by N) or MSE00a = mean(var(t',0)) % "a"djusted to be unbiased (divided by N-1)
whereas the biased and unbiased training MSEs are
MSE = sse(t-y)/Ntrneq MSEa = sse(t-y)/Ndof
7. The recommended training function for classification is 'trainscg'. A reasonable training goal is MSEgoal = 0.01*(Ndof/Ntreq)*MSE00a
8. Try a number of values for H and run 10 or more random weight initialization trials for each value of H..
>> 1. Increasing the inputclass database will improve but suggest me something other than that. > > 2. Increasing the number of hidden layers from 10 to many doesn't seem to make much difference :(.... > > Through the documentation of Neural Network toolbox - I found the default nprtool in Matlab take cares of input and output processing (ex: mapminmax) and also it uses trainscg function for training.... Should I use some other efficient training algorithms such as trainlm ? But here how can I decide logically (not by trying all algorithms) which training function will be useful for me.? > > I have just started to work on neural network after exploring some basics...Kindly help me on improving it -- making me a transition from advance beginner to expert :P > > Thanks for yourtime....!