Drexel dragonThe Math ForumDonate to the Math Forum



Search All of the Math Forum:

Views expressed in these public forums are not endorsed by Drexel University or The Math Forum.


Math Forum » Discussions » sci.math.* » sci.math.independent

Topic: Step Control for Gradient Descent ?
Replies: 5   Last Post: Feb 23, 2009 1:46 PM

Advanced Search

Back to Topic List Back to Topic List Jump to Tree View Jump to Tree View   Messages: [ Previous | Next ]
Dave Rudolf

Posts: 37
Registered: 10/3/05
Re: Step Control for Gradient Descent ?
Posted: Feb 23, 2009 1:46 PM
  Click to see the message monospaced in plain text Plain Text   Click to reply to this topic Reply

On Feb 18, 6:17 pm, Ray Vickson <RGVick...@shaw.ca> wrote:
> On Feb 18, 5:34 pm, dave.rud...@usask.ca wrote:
>
>
>

> > On Feb 18, 3:31 pm, rgvick...@gmail.com wrote:
>
> > > Why am I making an issue of this? Well, there are two aspects to your
> > > problem: (1) getting a local optimum in reasonable time and with
> > > reasonable accuracy; and (2) getting a global optimum. Let's just look
> > > at (1) for the moment. It has long been known through examples that
> > > simple gradient searches (of the type you seem to be using) are
> > > dangerous: you can have convergence to a NON-OPTIMAL point.

>
> > What do you mean non-optimal? Of course it can converge on a local
> > minimum, but that is a problem with any local method, as far as I
> > understand.

>
> By non-optimal, I mean not even a local minimum, even in a "convex"
> problem where any local minimum is automatically a global minimum. The
> successive points can get jammed up along a constraint, but at a point
> that is very far from satisfying optimality conditions.
>



Ah, I can see how penalty methods can help avoid this. Thanks.




Point your RSS reader here for a feed of the latest messages in this topic.

[Privacy Policy] [Terms of Use]

© Drexel University 1994-2014. All Rights Reserved.
The Math Forum is a research and educational enterprise of the Drexel University School of Education.