Search All of the Math Forum:

Views expressed in these public forums are not endorsed by NCTM or The Math Forum.

Notice: We are no longer accepting new posts, but the forums will continue to be readable.

Topic: the math in classify.m
Replies: 7   Last Post: Dec 3, 2012 7:34 PM

 Messages: [ Previous | Next ]
 Peter Perkins Posts: 1,665 Registered: 12/7/04
Re: the math in classify.m
Posted: Aug 9, 2005 9:48 AM

marquito wrote:

> =====================================================
> % Pooled estimate of covariance
> [Q,R] = qr(training - gmeans(gindex,:), 0);
> R = R / sqrt(n - ngroups); % SigmaHat = R'*R
> s = svd(R);
> if any(s <= eps^(3/4)*max(s))
> error('The pooled covariance matrix of TRAINING
> must be positive definite.');
> end
>
> % MVN relative log posterior density, by group, for
> each sample
> for k = 1:ngroups
> A = (sample - repmat(gmeans(k,:), mm, 1)) / R;
> D(:,k) = log(prior(k)) - .5*sum(A .* A, 2);
> end
> ======================================================
>
> I dont know exactly, was is going on there. I expected to see
> something like a multivariate Gauss distribution, like:
>
> p(x|class) = 1/sqrt(2pi^d * |Sigma| ) * exp( (x-mu)Sigma^-1(x-mu))
>
> or something similar to this. Could somebody verify this or explain
> what kind of magic the programmer used (why qr decomposition?).

You're right that the density formally involves the inverse of a cov matrix.
But inverting a potentially large matrix explicitly is usually not a good idea.
Write down what the estimate SigmaHat would be (X0'*X0, where X0 is the
centered data matrix, bearing in mind that the centering is different for each
class), then write X0 as Q*R. Just like the comment says, SigmaHat is R'*R,
because Q is orthonormal. Now substitute that into the quadratic form
(x-mu)Sigma^-1(x-mu), and you find that you can compute that quadratic form
using backsolve on a triangular R.

> I'm also very interested how 'D' is calculated since I use this value
> to show the distances of a sample to the different classes. Shouldn't
> this be the probability density function of x for the different
> classes?

It is the log of that, multiplied by the class prior probabilities. Computed on
the log scale, because multivariate probabilities get very, very small.

With a flat prior, I think the third output POSTERIOR is what you are asking for.

Hope this helps.

- Peter Perkins
The MathWorks, Inc.

Date Subject Author
8/9/05 marquito
8/9/05 Peter Perkins
8/9/05 marquito
8/9/05 Peter Perkins
8/9/05 Greg Heath
8/10/05 marquito
12/3/12 Aniket
12/3/12 Greg Heath