Search All of the Math Forum:

Views expressed in these public forums are not endorsed by NCTM or The Math Forum.

Topic: Non-linear optimization
Replies: 32   Last Post: Mar 8, 2013 2:22 AM

 Messages: [ Previous | Next ]
 Bruno Luong Posts: 9,240 Registered: 7/26/08
Re: Non-linear optimization
Posted: Mar 8, 2013 2:22 AM

"Matt J" wrote in message <khbsaf\$6jf\$1@newscl01ah.mathworks.com>...
> "Bruno Luong" <b.luong@fogale.findmycountry> wrote in message <khb5mn\$7bg\$1@newscl01ah.mathworks.com>...

> One can see that the empirical lambda-tuning rules in the original LM papers should in theory be applicable to true Hessians and not require an eig(H) operation. However, it's still easy to imagine that if the algorithm lands in a non-convex region where H is not positive definite, that you might have to solve
>
> (H+lambda*I)*x=-g
>
> for several lambda before a descent-direction was found.

Yes and usually the lambda is automatically adjusted (my multiplying by a constant factor) until criteria are fulfilled. Simple check as decrease of objective function will warrant (with probability) the lambda is large enough, even if H is not positive.

That having said, I don't know many people who actually are using the true Hessian in non-linear optimization.

Bruno

Date Subject Author
3/4/13 Toan Cao
3/4/13 Steven Lord
3/4/13 Toan Cao
3/5/13 Steven Lord
3/5/13 Toan Cao
3/6/13 Matt J
3/6/13 Matt J
3/6/13 Toan Cao
3/6/13 Matt J
3/4/13 Matt J
3/4/13 Toan Cao
3/5/13 Matt J
3/5/13 Bruno Luong
3/6/13 Matt J
3/6/13 Bruno Luong
3/6/13 Matt J
3/6/13 Bruno Luong
3/6/13 Matt J
3/6/13 Bruno Luong
3/6/13 Matt J
3/7/13 Bruno Luong
3/7/13 Matt J
3/7/13 Bruno Luong
3/7/13 Matt J
3/7/13 Bruno Luong
3/7/13 Matt J
3/7/13 Bruno Luong
3/8/13 Matt J
3/8/13 Bruno Luong
3/7/13 Toan Cao
3/7/13 Matt J
3/7/13 Toan Cao
3/7/13 Matt J