
Re: the math in classify.m
Posted:
Aug 9, 2005 9:48 AM


marquito wrote:
> ===================================================== > % Pooled estimate of covariance > [Q,R] = qr(training  gmeans(gindex,:), 0); > R = R / sqrt(n  ngroups); % SigmaHat = R'*R > s = svd(R); > if any(s <= eps^(3/4)*max(s)) > error('The pooled covariance matrix of TRAINING > must be positive definite.'); > end > > % MVN relative log posterior density, by group, for > each sample > for k = 1:ngroups > A = (sample  repmat(gmeans(k,:), mm, 1)) / R; > D(:,k) = log(prior(k))  .5*sum(A .* A, 2); > end > ====================================================== > > I dont know exactly, was is going on there. I expected to see > something like a multivariate Gauss distribution, like: > > p(xclass) = 1/sqrt(2pi^d * Sigma ) * exp( (xmu)Sigma^1(xmu)) > > or something similar to this. Could somebody verify this or explain > what kind of magic the programmer used (why qr decomposition?).
You're right that the density formally involves the inverse of a cov matrix. But inverting a potentially large matrix explicitly is usually not a good idea. Write down what the estimate SigmaHat would be (X0'*X0, where X0 is the centered data matrix, bearing in mind that the centering is different for each class), then write X0 as Q*R. Just like the comment says, SigmaHat is R'*R, because Q is orthonormal. Now substitute that into the quadratic form (xmu)Sigma^1(xmu), and you find that you can compute that quadratic form using backsolve on a triangular R.
> I'm also very interested how 'D' is calculated since I use this value > to show the distances of a sample to the different classes. Shouldn't > this be the probability density function of x for the different > classes?
It is the log of that, multiplied by the class prior probabilities. Computed on the log scale, because multivariate probabilities get very, very small.
With a flat prior, I think the third output POSTERIOR is what you are asking for.
Hope this helps.
 Peter Perkins The MathWorks, Inc.

