Search All of the Math Forum:
Views expressed in these public forums are not endorsed by
Drexel University or The Math Forum.
|
|
|
|
Re: Expectation Maximization Initialization
Posted:
Jul 11, 2006 5:19 AM
|
|
A.G.McDowell wrote:
> In article <44b233c4$1@news.nwl.ac.uk>, David Jones
<dajxxx@ceh.ac.uk>
> writes
>> dbmining@gmail.com wrote:
>>> How is the expectation maximization algorithm initialized? How
are
>>> the initial values estimated.
>>
>>
>> In principle this should not matter too much, so you would need to
>> judge how much to worry about being sophisticated. However
>> convergence can be slow.
>>
>> You can try setting missing values temporarily to a mean value,
>> although this will lead to an initially underestimated variance.
You
>> could use a second stage to this where missing values are replaced
by
>> random values generated from a first stage (poor) fitted model.
>>
>> You might find it beneficial to start from several different
initial
>> parameter sets in order to help you judge whether to stop
iterating.
>>
>> David Jones
>>
>>
> Does this advice stem from an assumption about the presence (or
rather
> absence) of multiple local minima?
No, it was rather about the possible slow convergence. Differences
between successive iterations might be compared with differences
between starting points as some guide to how long convergence might
take, with the possibility of restarting from some averaged value.
> I have seen Expectation
> Maximization considered in situations where the presence of multiple
> local minima was obvious (for example, to fit mixture models where
> the presence of a minimum at e.g. A=1, B=2 implied the existence of
> one at A=2, B=1). In this case you might try multiple starts from
> random initial parameters in the hope that at least one of your
> random starts would lead you to converge to a local minimum which
was
> also a global minimum. I am not sure what you could deduce about
> convergence from the different local minima found.
My limited experience was not in this context, and without multiple
maxima.
David Jones
|
|
|
|
