Date: May 10, 2010 1:01 PM
Author: phubaba
Subject: matrix differentials

Hello everyone,

I've been trying to learn matrix calculus in my free time, and I've been struggling with one general concept.

Given that you have a matrix differential example:

P = Nx1, V=NxN, V is a matrix of constants P is variable

then dP'VP
=d(P'V)P+P'd(VP)
=d(P')*VP + P'dV*P+P'dV*P + P'VdP
(second and third terms are 0 because dV is zero)
= d(P')*V*P + P'VdP

now my question is how do we manipulate the term d(P')*V*P . I know that the solution to the above derivative is: P'(V'+V). Therefore d(P')*V*P must equal P'V'd(P).

However, it is not clear to me how this is done.
http://ht.econ.kobe-u.ac.jp/~tanizaki/workshop/2007/20070309.pdf does something similar on page 23 example 5.2. This is also done in http://research.microsoft.com/en-us/um/people/minka/papers/matrix/minka-matrix.pdf on page 3.