Search All of the Math Forum:
Views expressed in these public forums are not endorsed by
Drexel University or The Math Forum.
|
|
mizhael
Posts:
139
Registered:
12/11/05
|
|
grid search for global optimum, is there a way to decide stepsize?
Posted:
Jun 25, 2006 9:22 PM
|
|
Hi all,
As you might have seen my previous posted questions, I am trying hard to
search for the global minimum of my 4D function z=f(x, y, u, v).
I have read literature extensively, and found out that there is no
algorithm/solver that is guaranteed to locate the global optimum.
So I decide to go back to the primitive grid search. I've already done a
0.02 step size 4D grid search. It was about 1 night's running time.
Now I want to ask: given this function is continuous except at a few
singularities(mainly due to log(0) and dividing by sigma=0), analytically I
may be able to obtain gradient functions and may be able to obtain the
Liptisz constant...
May I be able to decide the optimal stepsize that can guarantee to find the
global optimum?
I am thinking of if the optimal stepsize is not too small, and by adjusting
my bounds/search range using domain specific knowledge, I may be able to
find the global minimum using grid-search, after a few days waiting time...
(Slowness is better than missing the global optimum)...
Thanks a lot!
---------------------------------
Here is the description of the model, which fits data to an MLE log
likelihood function:
The model is of typical loglikelihood function:
f(mu, sigma, a, b)=-0.5*sum_of_squares( (x - mu)/sigma) + log(p1) - log(p0).
The log(p1) and log(p0) terms are the logs of some probabilities.
Theoretically all the above functions are continuous and well-behaved.
However, when fitting this model to data, due to model error or data noise,
the numerical results might not be always well-behaved.
Sigma being near 0 is a huge problem, although I've demanded the low bound
of sigma to be 1e-5.
p1 and p0 can be evaluated to be 0 or negative too. Although theoretically a
probability cannot be outside [0, 1], but when fitting it to data, it can
give negative values and values that are greater than 1, depending on the
data.
Thus this 4D log likelihood function is hard to maximize using data. I am
trying many solvers.
As you can see, theoretically, everything is continuous, except sigma=0, but
I've set lower bound of sigma to be 1e-5.
I even could not answer your question about "how many local minimums",
because with z=f(x, y, u, v) 4D function, I was unable to systematically
visualize the sample points at all.
Any thoughts?
|
|
|
|
