Drexel dragonThe Math ForumDonate to the Math Forum



Search All of the Math Forum:

Views expressed in these public forums are not endorsed by Drexel University or The Math Forum.


Math Forum » Discussions » sci.math.* » sci.stat.edu.independent

Topic: grid search for global optimum, is there a way to decide stepsize?
Replies: 18   Last Post: Jul 3, 2006 6:14 AM

Advanced Search

Back to Topic List Back to Topic List Jump to Tree View Jump to Tree View   Messages: [ Previous | Next ]
mizhael

Posts: 139
Registered: 12/11/05
grid search for global optimum, is there a way to decide stepsize?
Posted: Jun 25, 2006 9:22 PM
  Click to see the message monospaced in plain text Plain Text   Click to reply to this topic Reply

Hi all,

As you might have seen my previous posted questions, I am trying hard to
search for the global minimum of my 4D function z=f(x, y, u, v).

I have read literature extensively, and found out that there is no
algorithm/solver that is guaranteed to locate the global optimum.

So I decide to go back to the primitive grid search. I've already done a
0.02 step size 4D grid search. It was about 1 night's running time.

Now I want to ask: given this function is continuous except at a few
singularities(mainly due to log(0) and dividing by sigma=0), analytically I
may be able to obtain gradient functions and may be able to obtain the
Liptisz constant...

May I be able to decide the optimal stepsize that can guarantee to find the
global optimum?

I am thinking of if the optimal stepsize is not too small, and by adjusting
my bounds/search range using domain specific knowledge, I may be able to
find the global minimum using grid-search, after a few days waiting time...
(Slowness is better than missing the global optimum)...

Thanks a lot!

---------------------------------

Here is the description of the model, which fits data to an MLE log
likelihood function:


The model is of typical loglikelihood function:

f(mu, sigma, a, b)=-0.5*sum_of_squares( (x - mu)/sigma) + log(p1) - log(p0).

The log(p1) and log(p0) terms are the logs of some probabilities.

Theoretically all the above functions are continuous and well-behaved.

However, when fitting this model to data, due to model error or data noise,
the numerical results might not be always well-behaved.

Sigma being near 0 is a huge problem, although I've demanded the low bound
of sigma to be 1e-5.

p1 and p0 can be evaluated to be 0 or negative too. Although theoretically a
probability cannot be outside [0, 1], but when fitting it to data, it can
give negative values and values that are greater than 1, depending on the
data.

Thus this 4D log likelihood function is hard to maximize using data. I am
trying many solvers.

As you can see, theoretically, everything is continuous, except sigma=0, but
I've set lower bound of sigma to be 1e-5.

I even could not answer your question about "how many local minimums",
because with z=f(x, y, u, v) 4D function, I was unable to systematically
visualize the sample points at all.

Any thoughts?




Date Subject Author
6/25/06
Read grid search for global optimum, is there a way to decide stepsize?
mizhael
6/25/06
Read Re: grid search for global optimum, is there a way to decide stepsize?
John Herman
6/25/06
Read Re: grid search for global optimum, is there a way to decide stepsize?
mizhael
6/25/06
Read Re: grid search for global optimum, is there a way to decide stepsize?
Randy Poe
6/25/06
Read Re: grid search for global optimum, is there a way to decide stepsize?
C6L1V@shaw.ca
6/26/06
Read Re: grid search for global optimum, is there a way to decide stepsize?
mizhael
6/25/06
Read Re: grid search for global optimum, is there a way to decide stepsize?
Ron Baker, Pluralitas!
6/26/06
Read Re: grid search for global optimum, is there a way to decide stepsize?
mizhael
6/26/06
Read Re: grid search for global optimum, is there a way to decide stepsize?
Ron Baker, Pluralitas!
6/25/06
Read Re: grid search for global optimum, is there a way to decide stepsize?
jay
6/26/06
Read Re: grid search for global optimum, is there a way to decide stepsize?
mizhael
6/28/06
Read Re: grid search for global optimum, is there a way to decide stepsize?
David A. Heiser
6/29/06
Read Re: grid search for global optimum, is there a way to decide stepsize?
Ray Koopman
6/29/06
Read Re: grid search for global optimum, is there a way to decide stepsize?
David A. Heiser
6/29/06
Read Re: grid search for global optimum, is there a way to decide stepsize?
Phil Sherrod
6/29/06
Read Re: grid search for global optimum, is there a way to decide stepsize?
mizhael
7/1/06
Read Re: grid search for global optimum, is there a way to decide stepsize?
mizhael
7/3/06
Read Re: grid search for global optimum, is there a way to decide stepsize?
optionstraderjeff

Point your RSS reader here for a feed of the latest messages in this topic.

[Privacy Policy] [Terms of Use]

© Drexel University 1994-2014. All Rights Reserved.
The Math Forum is a research and educational enterprise of the Drexel University School of Education.