Drexel dragonThe Math ForumDonate to the Math Forum



Search All of the Math Forum:

Views expressed in these public forums are not endorsed by Drexel University or The Math Forum.


Math Forum » Discussions » Software » comp.soft-sys.matlab

Topic: Scaling of parameters for fminunc?
Replies: 26   Last Post: Jul 10, 2012 11:58 AM

Advanced Search

Back to Topic List Back to Topic List Jump to Tree View Jump to Tree View   Messages: [ Previous | Next ]
Luca

Posts: 77
Registered: 6/6/12
Re: Scaling of parameters for fminunc?
Posted: Jul 10, 2012 7:24 AM
  Click to see the message monospaced in plain text Plain Text   Click to reply to this topic Reply

"Bruno Luong" <b.luong@fogale.findmycountry> wrote in message
> When you scale the parameters, you affects the (1) non-linearity AND (2) the conditioning of the cost function. You can't overlook either aspect.

Thank you, very interesting.
I have to note that even without formal training on the topic I got right most of the points!
I have regularization, I spend lots of time in making a first guess which is "almost the solution", I compute the gradient (I took me a bit to get it correct :-p ) and so on.
The reason why the whole topic started is stated in this point in your old post:
"At east, the correct rescaling should be carried out on the unknown space so that the variation among unknowns must affect comparably on the cost function. Do not mix brutally parameters with different physical scales together."

The optimization worked correctly before scaling the parameters. But I knew that it wasn't correct to mix such different parameters, affecting much differently the cost function. Therefore I scaled my parameters hoping to improve the performance of the minimization. But it turned out the other way round! If I scale the parameters such that each unknown affects the cost function comparably... I get stuck in local minima!! (Worse to me than my initial guess!)
Without scaling some unknown are just untouched by the minimizator, they remain at the initial guess... And everything works better and local minima are avoided.



Point your RSS reader here for a feed of the latest messages in this topic.

[Privacy Policy] [Terms of Use]

© Drexel University 1994-2014. All Rights Reserved.
The Math Forum is a research and educational enterprise of the Drexel University School of Education.