Search All of the Math Forum:

Views expressed in these public forums are not endorsed by NCTM or The Math Forum.

Notice: We are no longer accepting new posts, but the forums will continue to be readable.

Topic: Which optimization method to use?
Replies: 5   Last Post: Dec 2, 2013 9:52 AM

 Messages: [ Previous | Next ]
 Tikkuhirvi Tietavainen Posts: 99 Registered: 4/22/08
Re: Which optimization method to use?
Posted: Nov 28, 2013 12:00 PM

Alan_Weiss <aweiss@mathworks.com> wrote in message <kj1pj5\$178\$1@newscl01ah.mathworks.com>...
> On 3/28/2013 11:23 AM, Aino wrote:
> > Alan_Weiss <aweiss@mathworks.com> wrote in message
> > <kj1dck\$hh5\$1@newscl01ah.mathworks.com>...

> >> On 3/27/2013 1:09 PM, Aino wrote:
> >> > Hello all.
> >> >
> >> > I have a function that includes a simulation. The input of the >

> >> simulation is seven parameters. I need to vary five of the seven >
> >> parameters to find the combination that would minimize the final >
> >> output (error that is calculated between a measured signal and the >
> >> simulated signal). I have somewhat a good initial guess for the five

> >> > parameters and I know that they should all be positive. I need to
> >> do > this optimization several times, so computational cost is an
> >> issue too.

> >> >
> >> > I tried the optimization with "fminsearch". It worked quite ok, but
> >> > two problems still occurred. First, one of the five parameters had
> >> > negative values. This particular parameter is possible to just

> >> leave > out. Second, it would seem that there are several local
> >> minimum that > are all feasible when considering the parameter values
> >> :( and > "fminsearch" didn't always find the global minimum. Also, I
> >> think that > the output error includes "noise", or maybe one/several
> >> parameters are > not always relevant.

> >> > I am now trying out "fminsearchbnd" (File Exchange) without the
> >> fifth > parameter and with lower bounds [0,0,0,0]. However, I was
> >> wondering if > "fmincon" would be more appropriate for me? I am not
> >> sure how to > implement it, though. Is there possibly even better
> >> (more robust, not > too time-consuming) optimization method for me?

> >> >
> >> > Thanks,
> >> > Aino

> >>
> >> You might be interested in the documentation on optimizing simulations:
> >> http://www.mathworks.com/help/optim/ug/optimizing-a-simulation-or-ordinary-differential-equation.html
> >>
> >>
> >> fmincon is indeed an appropriate solver for your problem. It is part
> >> of Optimization Toolbox. You should use the interior-point or sqp
> >> algorithms; see
> >> http://www.mathworks.com/help/optim/ug/writing-constraints.html#br9p_ry
> >>
> >> Alan Weiss
> >> MATLAB mathematical toolbox documentation

> >
> >
> > I am now testing the "fmincon" with "sqp". I still have some questions
> > about the optimization.. First, I don't know exactly the whole
> > parameter space (therefore I use only the "0" lower bounds and "Inf"
> > higher bounds) and with bad parameters the simulation stops and gives
> > something rather arbitrary for the error. I tried to put "NaN" as the
> > error when the simulation stops, but "sqp" algorithm doesn't seem to
> > know how to deal with the "NaN". Did I misunderstand the last part of
> > statement:
> >
> > "Objective function is undefined at initial point. Fmincon cannot
> > continue."
> >
> > Also, I would be happy with an error smaller than 0.05, is that what
> > "TolFun" is for? I didn't see any difference between TolFun=0.1 and
> > TolFun=0.05. Also, to speed the optimization, I would like to have a
> > minimum step size for the input parameters. "DiffMinChange"? Setting
> > DiffMinChange=0.01 lead to slower computing but also smaller error.
> > Also, the parameters have a very different magnitude, is it possible
> > to tweak the step size separately for each parameter? I have to say
> > I'm a bit lost with this optimization.
> >
> > -Aino

>
> If the simulation fails at the initial point x0, then fmincon fails, and
> there is nothing you can do about it except ensure that the initial
> point leads to a sensible result.
>
> To find out what the various tolerances such as TolFun mean, see
> http://www.mathworks.com/help/optim/ug/tolerances-and-stopping-criteria.html
>
> To see what the DiffMinChange option means, read carefully the section
> http://www.mathworks.com/help/optim/ug/optimizing-a-simulation-or-ordinary-differential-equation.html
>
> You might also want to look at the options description:
> http://www.mathworks.com/help/optim/ug/optimization-options-reference.html
>
> The "step size" might not mean what you think it means. You seem to
> think it is the change in parameters that fmincon takes. But it is, in
> fact, the size of a step for estimating gradients by finite differences.
>
> Good luck,
>
> Alan Weiss
> MATLAB mathematical toolbox documentation

Hello again. :)

I'm still working with this optimization problem, though the simulation is updated a bit. I realized another possible problem. Changing the input parameters e.g. by 0.05 (5%), some parameters have larger effect to the outcome measure (the variable that is being minimized) than others. Could this have an effect to my results (optimized parameters)?

-Aino

Date Subject Author
3/28/13 Alan Weiss
3/28/13 Tikkuhirvi Tietavainen
3/28/13 Alan Weiss
11/28/13 Tikkuhirvi Tietavainen
12/2/13 Alan Weiss