> >It looks like finding lambda for quasi-Newton-LM would be much more efficient than for true Newton-LM. > > Not sure I understand your statement. In LM method, finding lambda is based on the same empirical rules that works equally well regardless which the Hessian approximation or true Hessian is chosen. =================
I don't see how that can be. For non-convex functions and non-posdef Hessians, an empirically chosen lambda could easily leave H+lambda*I singular, or at least, not positive definite and therefore non-descending. You would have to choose lambda>=min(eig(H)) to be sure that didn't happen, and that would require an eigen-analysis of H.