> ===================================================== > % Pooled estimate of covariance > [Q,R] = qr(training - gmeans(gindex,:), 0); > R = R / sqrt(n - ngroups); % SigmaHat = R'*R > s = svd(R); > if any(s <= eps^(3/4)*max(s)) > error('The pooled covariance matrix of TRAINING > must be positive definite.'); > end > > % MVN relative log posterior density, by group, for > each sample > for k = 1:ngroups > A = (sample - repmat(gmeans(k,:), mm, 1)) / R; > D(:,k) = log(prior(k)) - .5*sum(A .* A, 2); > end > ====================================================== > > I dont know exactly, was is going on there. I expected to see > something like a multivariate Gauss distribution, like: > > p(x|class) = 1/sqrt(2pi^d * |Sigma| ) * exp( (x-mu)Sigma^-1(x-mu)) > > or something similar to this. Could somebody verify this or explain > what kind of magic the programmer used (why qr decomposition?).
You're right that the density formally involves the inverse of a cov matrix. But inverting a potentially large matrix explicitly is usually not a good idea. Write down what the estimate SigmaHat would be (X0'*X0, where X0 is the centered data matrix, bearing in mind that the centering is different for each class), then write X0 as Q*R. Just like the comment says, SigmaHat is R'*R, because Q is orthonormal. Now substitute that into the quadratic form (x-mu)Sigma^-1(x-mu), and you find that you can compute that quadratic form using backsolve on a triangular R.
> I'm also very interested how 'D' is calculated since I use this value > to show the distances of a sample to the different classes. Shouldn't > this be the probability density function of x for the different > classes?
It is the log of that, multiplied by the class prior probabilities. Computed on the log scale, because multivariate probabilities get very, very small.
With a flat prior, I think the third output POSTERIOR is what you are asking for.