Search All of the Math Forum:
Views expressed in these public forums are not endorsed by
Drexel University or The Math Forum.
|
|
|
|
Re: regularization?
Posted:
Feb 23, 2012 4:45 AM
|
|
On 22/02/2012 20:45, RichD wrote:
> On Feb 16, Martin Brown<|||newspam...@nezumi.demon.co.uk> wrote:
>>> What is regularization, in applied math and stats?
>>
>> In this context some function of the reconstructed
>> model that can be used as a constraint on possible
>> solutions. In effect steering the solution
>> towards desirable properties based on a priori
>> knowledge.
>
> So it's a constraint on the solution space,
> not the domain space?
It is used to encode prior knowledge and this is usually a constraint on
the solution space like positivity or smoothness for example.
>
> It it used exclusively for underdetermined
> systems?
It would be redundant for a system of equations that already had a
unique solution or was over determined to begin with.
>> For instance the sky is everywhere positive is a
>> rather powerful strong
>> constraint in astronomical deconvolution programmes.
>
> ?
Have you ever seen negative sky brightness?
Many simple linear methods of deconvolution like Wiener filters produce
them. They are a strong hint that the reconstruction is physically
impossible. Barrier functions provide a convenient way to enforce the
positivity constraint on the set of possible solutions.
>> This has been encoded in various forms such as
>> maximum entropy sum(log(f)) by Burg or
>> sum(flog(f)) by Gull& Daniell. You can choose
>> various other regularising functions which will
>> introduce different biasses in the final solution.
>> sum(f^2) or sum(|f"|) for instance.
>
> I don't understand what these (f) things are,
> or how they enter the calculations.
f[x,y] is the reconstructed brightness solution as a function of
position (or whatever it is you are trying to determine).
>
>> Gull demonstrated it nicely in his Kangaroos problem. see
>>
>> http://sci.tech-archive.net/Archive
>> /sci.image.processing/2008-10/msg00099.html
>
> It might be a nice demonstration nicely, but
> the article is poorly written.
In what way? Here is a link to Jaynes version:
http://bayes.wustl.edu/etj/articles/cmonkeys.pdf
>> ISTR The original paper is called Monkeys,
>> Kangeroos and N.
>>
>>> Is it synonymous with optimization?
>>> I recently heard a speaker refer to it as
>>> a means of achieving algorithmic
>>> and noise robustness, but I didn't get it.
>>
>> A suitable choice of regularising function can make
>> an otherwise ill posed problem tractable and/or
>> have a unique solution.
>
> It's not clear to me, whether the regularizer
> is applied to the data as a pre-processing step,
> or as an extra set of equations to be satisfied.
It is an extra set of regularising constraints on the problem.
Minimise C = sum[ ((data - model)/sigma)^2)
subject to additional regularising function
Maximise S = sum[ -f.log(f) ]
Look up Lagrange multipliers.
--
Regards,
Martin Brown
|
|
|
|
