As you might have seen my previous posted questions, I am trying hard to search for the global minimum of my 4D function z=f(x, y, u, v).
I have read literature extensively, and found out that there is no algorithm/solver that is guaranteed to locate the global optimum.
So I decide to go back to the primitive grid search. I've already done a 0.02 step size 4D grid search. It was about 1 night's running time.
Now I want to ask: given this function is continuous except at a few singularities(mainly due to log(0) and dividing by sigma=0), analytically I may be able to obtain gradient functions and may be able to obtain the Liptisz constant...
May I be able to decide the optimal stepsize that can guarantee to find the global optimum?
I am thinking of if the optimal stepsize is not too small, and by adjusting my bounds/search range using domain specific knowledge, I may be able to find the global minimum using grid-search, after a few days waiting time... (Slowness is better than missing the global optimum)...
Thanks a lot!
Here is the description of the model, which fits data to an MLE log likelihood function:
The log(p1) and log(p0) terms are the logs of some probabilities.
Theoretically all the above functions are continuous and well-behaved.
However, when fitting this model to data, due to model error or data noise, the numerical results might not be always well-behaved.
Sigma being near 0 is a huge problem, although I've demanded the low bound of sigma to be 1e-5.
p1 and p0 can be evaluated to be 0 or negative too. Although theoretically a probability cannot be outside [0, 1], but when fitting it to data, it can give negative values and values that are greater than 1, depending on the data.
Thus this 4D log likelihood function is hard to maximize using data. I am trying many solvers.
As you can see, theoretically, everything is continuous, except sigma=0, but I've set lower bound of sigma to be 1e-5.
I even could not answer your question about "how many local minimums", because with z=f(x, y, u, v) 4D function, I was unable to systematically visualize the sample points at all.