i have fear of asking this question, because it could be stupid, but i will try it nevertheless:
suppose i have a matrix A N x 8, where N is of order 10^3-10^5 and a vector v of size Nx1 (all are real elements).
To compute the element (i, j) of A i need to do some computation (derivative of image pixels, some multiplications and so on). After computing A, i need to compute its pseudo-inverse: pinvA = (A^T * A)^-1 * A^T (where A^T is the transpose of A and (..)^-1 is the inversione of the quantity between parenthesis) and finally i need to compute the multiplication of the pesudoinverse by v:
pInvA * v
The result will be a vector x0 of size 8 x 1.
Here are the sequence of steps i do:
1] compute A 2] multiply A^T * A, without forming A^T (this step require 64*N double floating point multiplications) 3] compute the inverse of A^T * A with the LU decomposition (A^T * A is a 8x8 matrix, so its inversion is not so slow to compute). let B = (A^T * A)^-1 4] compute v1 = A^T * v 5] compute x0 = B * v1
The step 2] require a lot of work. Is there a way to form A^T * A without doing so much multiplications?!??!
I have another question. A^T * A is a ill-conditioned matrix (its condition number is of order O(10^11)). Surprising enough, the algorithm is working reasonably well but sometimes fails. I am trying to understand the source of the fails...What i am asking is: O(10^11) is really a bad condition number, right? But if so, why it works at all?!??? Should i seriously address this "problem" or not? How can i lower the condition number of a large matrix?