Search All of the Math Forum:

Views expressed in these public forums are not endorsed by NCTM or The Math Forum.

Topic: Neural Network -- Incremental Training
Replies: 8   Last Post: Jun 5, 2014 11:41 PM

 Messages: [ Previous | Next ]
 Greg Heath Posts: 6,311 Registered: 12/7/04
Re: Neural Network -- Incremental Training
Posted: May 29, 2014 5:46 AM

"Marko Kolarek" <kolarek@gmail.com> wrote in message <lm4ms9\$ejq\$1@newscl01ah.mathworks.com>...
> "George Xu" <gxu5@jhu.edu> wrote in message <ef314ef.1@webx.raydaftYaTP>...
> > Thank you for your input Greg. I'll give this a shot.
> >
> > Thanks again!
> >
> > Greg Heath wrote:

> > >
> > >
> > > George Xu wrote:

> > >> Hello,
> > >>
> > >> I would like to use the Neural Network module to train a

> > network
> > > that
> > >> can be used as a classifier. At a later time when additional
> > data
> > > is
> > >> available, train the network further using the new data but
> > > without
> > >> training everything else again.
> > >>
> > >> Is there a way to load an existing network, perform the

> > training,
> > > and
> > >> add the weights from the new training to the existing network?
> > >
> > > You have to
> > >
> > > 1. Prevent the network from "forgetting the old data" by saving
> > > a Calibration Set which will be combined with the new data
> > > when the net is retrained.
> > > 2. Decide on the relative weighting of the old and new data.
> > >
> > > The best way is to use an RBF (help newrb). However, you might
> > > run into the problem of having more estimated parameters than
> > > you have training equations (Neq = Ntrn*O = product of number
> > > of training cases and number of network outputs). You then have
> > > to write some original code to do one or more of the following
> > >
> > > 1. Remove some redundant RBFs
> > > 2. Merge some neighboring RBFs
> > > 3. Determine the output layer weights using pinv.
> > >
> > > I think you can do 3 by modifying newrb.
> > >

> > >> My current implementation, which does not seem to work is as
> > > follows:
> > >>
> > >> [NewNet, tr] = train(OldNet, features)

> > >
> > > Aha! The classical "plasticity/stability dilema" which should have
> > > been named "plasticity/stability tradeoff". If you stick with an
> > > MLP
> > > (help newff) I can't think of a good fix.
> > >

> > >> Any help would be greatly appreciate it.
> > >>
> > >> Thanks!

> > >
> > > The problem is solvable. However, it will take a good understanding
> > > of the newrb source code before you can begin to deal with
> > > mitigating the overfitting problem of more parameters to
> > > estimate than you have training equations.
> > > with new data.
> > >
> > > Good Luck.
> > >
> > > Greg
> > >
> > >

>
> I am sorry to be reviving this old thread, but could you please tell me have you come up with a solution?

I have described the solution above and coded tens of versions in Fortran from 1983 to
1998. However, they were government sponsored and are not available.

A post-retirement MATLAB version in 2004 was lost in a computer crash. I have no desire to try to reconstruct it.

Date Subject Author
4/19/06 George Xu
4/22/06 Greg Heath
4/24/06 George Xu
5/28/14 Marko Kolarek
5/29/14 Greg Heath
5/30/14 Marko Kolarek
5/31/14 Greg Heath
6/5/14 Marko Kolarek
6/5/14 Greg Heath