Matrix

From Math Images

(Difference between revisions)
Jump to: navigation, search
(Matrix Multiplication)
(Matrix Transposition)
Line 389: Line 389:
====Properties of Matrix Transposition====
====Properties of Matrix Transposition====
-
#'''<balloon title="For any matrix A of no size restriction, then the transpose of its transpose is simply itself.">Self-inverse</balloon> of the transpose'''<math>(A^T)^T=A</math>
+
#'''<balloon title="A function which, when applied to an object twice, returns the original object.">Self-inverse</balloon> of the transpose'''<math>(A^T)^T=A</math>
-
#'''Transposes preserve <balloon title= "For any matrices A and B of the same size, the transpose of their matrix sum is equivalent to the sum of the individual matrices transposed">addition</balloon>'''<math>(A+B)^T=A^T+B^T</math>
+
#'''Transposes preserve addition''' <math>(A+B)^T=A^T+B^T</math>
-
#'''Transposes reverse the <balloon title= "For matrix A of size ''m x n'' and matrix B of size ''n x p'' in that order, the transpose of their matrix product is equivalent to the product of those separately transposed matrices in reverse order.">ordering</balloon> of the matrices'''<math>(AB)^T=B^TA^T </math>
+
#'''Transposes reverse the ordering of the matrices''' <math>(AB)^T=B^TA^T </math>
-
#'''Transposes preserve <balloon title= "For any matrix A of no size restriction and scalar value ''k'', the transpose of its product, kA, is equivalent to the product of the scalar and the transpose of matrix A. ">scalar multiplication</balloon>''' <math>(kA)^T=kA^T</math>, where <math> k</math> is a scalar multiple.
+
#'''Transposes preserve scalar multiplication''' <math>(kA)^T=kA^T</math>, where <math> k</math> is a scalar multiple.
====Proofs of Matrix Transposition Properties====
====Proofs of Matrix Transposition Properties====
Line 398: Line 398:
'''1. Proof that <math>(A^T)^T=A</math>'''
'''1. Proof that <math>(A^T)^T=A</math>'''
-
By the definition of matrix equality, we want to show (WTS) that the corresponding entries for the transpose of matrix A transposed equal the corresponding entries of the original matrix A.
+
By the definition of matrix equality, we want to show that the entries for the transpose of the transpose of matrix A equals the corresponding entries of the original matrix A:
-
:::::<math>WTS: [(A^T)^T]_{ij}= A_{ij}</math>
+
:::::<math>Claim: ((A^T)^T)_{ij}= A_{ij}</math>
-
Working from the left hand side (LHS) of the proposition, we apply the definition of matrix transpose to <math> A^T </math>:
+
Working from the left hand side of the proposition, we apply the definition of matrix transpose to <math> A^T </math>:
-
:::::<math> [(A^T)^T]_{ij}= (A^T)_{ji} </math>
+
:::::<math> ((A^T)^T)_{ij}= (A^T)_{ji} </math>
-
Then, applying the definition of the matrix transpose again, but to <math>A </math>:
+
Then, applying the definition of the matrix transpose to the above result:
:::::<math>(A^T)_{ji} = A_{ij} </math>
:::::<math>(A^T)_{ji} = A_{ij} </math>
-
We have manipulated the LHS so that it equals the RHS, the corresponding entries of original matrix A. The notion of matrix equality holds so the proof is complete.
+
We have manipulated the left hand side so that it equals the right hand side, the corresponding entries of original matrix A. The notion of matrix equality holds so the proof is complete.
-
*Note: Even without the proof, this transpose property makes intuitive sense because say, if you imagine taking the transpose twice as flipping over a diagonal twice, then you are back where you started.
+
Even without the proof, this transpose property makes intuitive sense. Imagine taking the transpose twice as flipping a diagonal twice, which takes you back where you started.
'''2. Proof that <math>(A+B)^T=A^T+B^T</math>'''
'''2. Proof that <math>(A+B)^T=A^T+B^T</math>'''
-
By the definition of matrix equality, we want to show (WTS) that the corresponding entries for the transpose of the summed matrix (A+B) is equivalent to the corresponding entries for the sum of individually transposed matrices.
+
By the definition of matrix equality, we want to show that the entries of the transpose of the matrix (A+B) are equivalent to the corresponding entries for the sum of the individual transposes of A and B.
-
:::::<math>WTS: ((A+B)^T)_{ij}= (A^T + B^T)_{ij} </math>
+
:::::<math>Claim: ((A+B)^T)_{ij}= (A^T + B^T)_{ij} </math>
-
Working from the left hand side (LHS) of the proposition, we apply the definition of matrix transposition to the summed matrix (A+B):
+
Working from the left hand side of the proposition, we apply the definition of matrix transposition to the summed matrix (A+B):
:::::<math> ((A+B)^T)_{ij}= (A+B)_{ji} </math>
:::::<math> ((A+B)^T)_{ij}= (A+B)_{ji} </math>
Line 427: Line 427:
By the definition of matrix addition, we further reduce the matrix (A+B) down to its real number entries:
By the definition of matrix addition, we further reduce the matrix (A+B) down to its real number entries:
-
:::::<math> (A+B)_{ji}= a_{ji} + b_{ji} </math>
+
:::::<math> (A+B)_{ji}= A_{ji} + B_{ji} </math>
We have reduced the left hand side of the proposition as much as possible.
We have reduced the left hand side of the proposition as much as possible.
-
Now, working from the right hand side (RHS) of the proposition, we apply the definition of matrix addition to the individual transposed matrices, <math>A^T</math>, and <math>B^T</math>:
+
Now, working from the right hand side of the proposition, we apply the definition of matrix addition to the individual transposed matrices, <math>A^T</math>, and <math>B^T</math>:
:::::<math> (A^T + B^T)_{ij} = (A^T)_{ij} + (B^T)_{ij} </math>
:::::<math> (A^T + B^T)_{ij} = (A^T)_{ij} + (B^T)_{ij} </math>
-
By the definition of the matrix transpose applied to matrices A and B, we reduce them down to their respective real number entries. We arrive at the same reduction that we obtained while working from the LHS.
+
By the definition of the matrix transpose applied to matrices A and B, we arrive at the same reduction that we obtained while working from the left hand side.
-
:::::<math>(A^T)_{ij} + (B^T)_{ij} = a_{ji} + b_{ji} </math>
+
:::::<math>(A^T)_{ij} + (B^T)_{ij} = A_{ji} + B_{ji} </math>
Because <math>LHS = RHS </math>, the notion of matrix equality holds. The proof is complete.
Because <math>LHS = RHS </math>, the notion of matrix equality holds. The proof is complete.
Line 445: Line 445:
'''3. Proof that <math> (AB)^T= B^TA^T </math>'''
'''3. Proof that <math> (AB)^T= B^TA^T </math>'''
-
By the definition of matrix equality, we want to show (WTS) that the corresponding entries for the transpose of the matrix product equals the corresponding entries of the product of separately transposed matrices in reverse order.
+
By the definition of matrix equality, we want to show that the entries for the transpose of the product of two matrices equal the corresponding entries of the product of separately transposed matrices in reverse order.
-
:::::<math>WTS: ((AB)^T)_{ij}=(B^TA^T)_{ij}</math>
+
:::::<math>Claim: ((AB)^T)_{ij}=(B^TA^T)_{ij}</math>
-
Starting from the left hand side (LHS) of the proposition, we apply the definition of the matrix transpose to the matrix, (AB)
+
Starting from the left hand side of the proposition, we apply the definition of the matrix transpose to the matrix, (AB)
:::::<math>((AB)^T)_{ij} = (AB)_{ji} </math>
:::::<math>((AB)^T)_{ij} = (AB)_{ji} </math>
Line 455: Line 455:
By the definition of matrix multiplication applied to the matrix (AB):
By the definition of matrix multiplication applied to the matrix (AB):
-
:::::<math> (AB)_{ji}=\sum_{k=1}^na_{jk}b_{ki} </math>
+
:::::<math> (AB)_{ji}=\sum_{k=1}^nA_{jk}B_{ki} </math>
We have reduced the left hand side of the proposition as much as possible, down to the real numbered entries of matrix (AB).
We have reduced the left hand side of the proposition as much as possible, down to the real numbered entries of matrix (AB).
-
Now working from the right hand side (RHS) of the proposition, we apply the definition of matrix multiplication to the matrices <math>B^T</math> and <math>A^T</math>:
+
Now working from the right hand side of the proposition, we apply the definition of matrix multiplication to the matrices <math>B^T</math> and <math>A^T</math>:
:::::<math> (B^TA^T)_{ij}= \sum_{k=1}^n(B^T)_{ik}(A^T)_{kj} </math>
:::::<math> (B^TA^T)_{ij}= \sum_{k=1}^n(B^T)_{ik}(A^T)_{kj} </math>
Line 465: Line 465:
By the definition of matrix transpose applied to matrices B and A:
By the definition of matrix transpose applied to matrices B and A:
-
:::::<math>\sum_{k=1}^n(B^T)_{ik}(A^T)_{kj} =\sum_{k=1}^n b_{ki}a_{jk} </math>
+
:::::<math>\sum_{k=1}^n(B^T)_{ik}(A^T)_{kj} =\sum_{k=1}^n B_{ki}A_{jk} </math>
-
Now we have reduced the matrices of B and A to their real number entries. So by the commutative property of real numbers under multiplication:
+
Now we have reduced the matrices of B and A to their real number entries. By the commutative property of real numbers under multiplication:
-
:::::<math>\sum_{k=1}^n b_{ki}a_{jk} =\sum_{k=1}^n a_{jk}b_{ki} </math>
+
:::::<math>\sum_{k=1}^n B_{ki}A_{jk} =\sum_{k=1}^n A_{jk}B_{ki} </math>
-
Now, <math>LHS = RHS </math>. The notion of matrix equality holds and the proof is complete.
+
Now, we have shown that the left hand side equals the right hand side of our original claim. The notion of matrix equality holds. Thus, transposes reverse the ordering of the matrices.
'''4. Proof that <math>(kA)^T=kA^T</math>'''
'''4. Proof that <math>(kA)^T=kA^T</math>'''
-
By the definition of matrix equality, we want to show (WTS) that the corresponding entries for the transpose of a scalar matrix product equals the corresponding entries for the product of a scalar and the matrix transpose.
+
By the definition of matrix equality, we want to show that the entries for the transpose of a scalar matrix product equal the corresponding entries for the product of a scalar and the matrix transpose.
-
:::::<math> WTS: ((kA)^T)_{ij}=(kA^T)_{ij} </math>
+
:::::<math> Claim: ((kA)^T)_{ij}=(kA^T)_{ij} </math>
-
Working from the left hand side (LHS) of the proposition, we apply the definition of the matrix transpose to matrix (kA):
+
Working from the left hand side of the proposition, we apply the definition of the matrix transpose to matrix (kA):
:::::<math> ((kA)^T)_{ij}= (kA)_{ji}</math>
:::::<math> ((kA)^T)_{ij}= (kA)_{ji}</math>
-
We have reduced matrix (kA) down to its real number entries so by multiplication of real numbers, we obtain:
+
By the properties of multiplying a scalar with a matrix, we obtain:
:::::<math>(kA)_{ji}=k A_{ji}</math>
:::::<math>(kA)_{ji}=k A_{ji}</math>
-
We have reduced the LHS as much as possible. Now moving onto the right hand side (RHS) of the proposition.
+
We have reduced the left hand side as much as possible. Now moving onto the right hand side of the claim.
-
The right hand side of the proposition already features real number entries, so by the multiplication of real numbers applied to the entries of the matrix (kA^T):
+
By the properties of multiplying a scalar with a matrix applied to the entries of the matrix (kA^T):
:::::<math>(kA^T)_{ij}= k (A^T)_{ij}</math>
:::::<math>(kA^T)_{ij}= k (A^T)_{ij}</math>
Line 498: Line 498:
:::::<math>k (A^T)_{ij}=k A_{ji} </math>
:::::<math>k (A^T)_{ij}=k A_{ji} </math>
-
Finally, <math>LHS = RHS </math>. The notion of matrix equality holds and the proof is complete.
+
We have shown that the reduction of the left hand side equals the right hand side and vice versa. The notion of matrix equality holds. Thus, transposes preserve scalar multiplication.
}}
}}

Revision as of 15:54, 26 July 2013

This is a Helper Page for:
Blue Fern
Summation Notation
Change of Coordinate Systems
Math for Computer Graphics and Computer Vision

A matrix is a rectangular array of numbers that can be used to store numbers for later access. In this helper page, we will discuss the mathematical properties of matrices.

Contents

Size

A matrix is typically described in terms of its size. If a Matrix M has m rows and n columns, we say that M has size m x n, or more simply, we say M is m x n. The numbers m and n are sometimes called the dimensions of M.

Matrix Notation

Matrices and their entries are closely related. To avoid confusion, separate notation exists for both matrices and entries.

Before we begin, we must introduce the general notation used to indicate a location of an entry in a matrix. For a matrix A,  A_{ij} indicates the entry located in the ith row and jth column. Below is matrix A with entries whose row and column locations are explicitly shown.


\begin{pmatrix}
a_{11} & a_{12}& \cdots & a_{1j}\\
a_{21} &  a_{22}& \cdots & a_{2j}\\
\vdots  & \vdots  & \ddots & \vdots  \\
a_{i1} & a_{i2} & \cdots & a_{ij}
 \end{pmatrix}

1.Notation for a Matrix:  A=[a_{ij}]

In other words, a matrix can be defined by just a capital letter or by the genral entry inside brackets.
  • A matrix is the same thing as its general entry enclosed in brackets.

2.Notation for an Entry:  a_{ij}=(A)_{ij}

In other words, an entry, as typically represented as a lowercase letter with a ij subscript, can also be written by enclosing the name of the matrix with parenthesis. This convention applies even to matrices such as C(BA) which has the typical entry denoted as (C(BA))_{ij}
  •  A_{ij} and a_{ij}are two ways of saying the same thing.

Matrix Equality

In order to prove identities about matrices, we need to first define what it means for two matrices to be equal!

Definition: Two matrices A and B are equal and we write A=B, if they are the same size and a_{ij}=b_{ij} for all i rows and j columns.

In other words, matrices are equal if they are of the same size and have the same corresponding entries.

In proving identities and properties of matrices, this definition comes in handy because the proofs rely on proving that the general entry of one matrix equals to the general entry of the other (a_{ij}=b_{ij}), which would therefore mean that both matrices is equal if their sizes are equal.

Matrix Operations

There are three fundamental matrix operations: addition, scalar multiplication, and matrix multiplication. Properties for each operation are given below along with proofs.

Matrix Addition

For matrix addition, the sum of two matrices  A = [a_{ij}] and  B = [b_{ij}] must have the same size in which case their matrix sum is defined as A + B = [a_{ij} + b_{ij}]. That is, each entry in the A+B matrix is the sum of the corresponding entries in the separate matrices. For instance:

 \begin{bmatrix}
          a & b \\
          c & d \\
         \end{bmatrix}
+        \begin{bmatrix}
          e & f \\
          g & h \\
         \end{bmatrix}
=        \begin{bmatrix}
          a+e & b+f \\
          c+g & d+h\\
         \end{bmatrix}

Properties of Matrix Addition

  1. Addition is Commutative:A+B=B+A
  2. Addition is Associative:(A+B)+C=A+(B+C)
  3. Identity of Addition:A+0=A

Proofs for properties of Matrix Addition

1. Proof for Commutativity of Addition: A+B=B+A

By the definition of matrix equality, we want to show that the corresponding entries of A+B and B+A are the same:

Claim: (A+B)_{ij}=(B+A)_{ij}

We work from the left hand side of the above proposition and attempt to prove that it equals the right hand side of the proposition. So starting from the left, we apply the definition of matrix addition to A+B, obtaining:

(A+B)_{ij}= A_{ij} + B_{ij}

Notice that we have reduced matrices A and B down to their real number entries, so the commutativity of real numbers applies. That is:

A_{ij}+B_{ij}=B_{ij} + A_{ij}

Lastly, by the definition of matrix addition applied to B and A:

B_{ij} + A_{ij}=(B+A)_{ij}

Stringing the last few equalities together, we get that:

 (A+B)_{ij}=(B+A)_{ij}

That is, we have shown that the left hand side of the original proposition equals its right hand side. Thus, matrix addition is commutative.


2. Proof for Associativity of Addition: (A+B) + C = A + (B+C)

By the definition of matrix equality, we want to show that the entries of (A+B)+ C and A+(B+C) are the same. That is:

 Claim: ((A+B) + C)_{ij} = (A + (B+C))_{ij}

We work from the left hand side of the above proposition and attempt to prove that it equals the right hand side of the proposition. Starting from the left, we apply the definition of matrix addition to the big composite matrix ((A+B)+C), breaking it down into two separate matrices, (A+B) and C.

 ((A+B)+C)_{ij}=(A+B)_{ij} + C_{ij}

Then, we break down the matrix (A+B) into the separate matrices A and B into the real number entries of A and B by the definition of matrix addition:

(A+B)_{ij} + C_{ij}=(A_{ij} + B_{ij})+ C_{ij}

Because we are now dealing with real number entries, the associativity of real numbers applies to the entries of matrices A, B, and C:

(A_{ij} + B_{ij})+ C_{ij} =A_{ij} + (B_{ij} + C_{ij})

Notice that the right hand side of the above equality includes the sum of the ijth entries for each of the B and C matrices, which is actually just the ijth entry of B+C:

A_{ij} + (B_{ij} + C_{ij})=A_{ij} + (B+C)_{ij}

Lastly, by the definition of matrix addition applied to B+C and A:

A_{ij} + (B+C)_{ij}=(A+ (B+C))_{ij}
Referring back to the original proposition, we have proved that its LHS equals the RHS, thus completing the proof.

Scalar Multiplication

Definition: For a m \times n matrix  A = [a_{ij}] and a scalar value k, the scalar product kA is defined by  kA=[ka_{ij}], where kA is also  m \times n.

The Entry definition is defined as:  (kA)_{ij} = ka_{ij}

For example:

 2 \begin{bmatrix} 
                   2 & 4 \\
                   0 & -0.3
                  \end{bmatrix} 
=                 \begin{bmatrix} 
                   4 & 8 \\
                   0 & -0.6
                  \end{bmatrix}

Properties of Scalar Multiplication

  1. k(A+B)=kA+kB
  2. (cd)A=c(dA), where c and d are real numbers.

Proofs of Scalar Multiplication Properties

1. Proof of Property 1: k(A+B)=kA+kB, where A and B are the same size.

First, by the definition of matrix equality, we want to show (WTS) that the corresponding entries of the composite matrix (A+B) times a scalar is equal to the corresponding entries of the composite matrix kA+kB:

WTS: (k(A+B))_{ij}=(kA+kB)_{ij}

We work from the left hand side (LHS) of the above proposition and attempt to prove that it equals the right hand side (RHS) of the proposition. So starting from the LHS, we apply the formal definition of Scalar Multiplication to the composite matrix k(A+B):

(k(A+B))_{ij}=k(A+B)_{ij}

Now we can apply the definition of matrix addition to A and B

k(A+B)_{ij}=k((A)_{ij}+(B)_{ij})

Since we are left with only real numbers now ((A)_{ij} is an entry), we can apply the distributive property of real numbers

k((A)_{ij}+(B)_{ij})=kA_{ij}+kB_{ij}

We work from the right hand side (RHS) of the initial proposition to see if it reduces to the last simplification of the left hand side. We see that the right hand side can use some splitting up (because of the entry notation), so we use the definition of matrix addition applied to the matrix kA+kB:

(kA+kB)_{ij}=(kA)_{ij}+(kB)_{ij}

Now we see scalars separately attached to matrices and should immediately consider applying the definition of scalar multiplication, which should be done next to matrices kA and kB

(kA)_{ij}+(kB)_{ij}=k(A)_(ij)+k(B)_{ij}

Since the entries of k(A+B) equal the entries of kA+kB, or LHS=RHS, and under the assumption that they are the same size, we can conclude that the matrices are equal from the definition of matrix equality.


2. Proof of Property 2: ((cd)A))_{ij}=(c(dA))_{ij}

First, by the definition of matrix equality, we want to show (WTS) that the corresponding entries of the matrix A times the quantity of the scalar c times d is equal to the corresponding entries of a scalar c times the quantity of a matrix A times a scalar, d:

We work from the left hand side (LHS) of the above proposition and attempt to prove that it equals the right hand side (RHS) of the proposition. So starting from the LHS, we apply the formal definition of Scalar Multiplication to (cd)A:

((cd)A)_{ij}=(cd)a_{ij}

We can leave the left hand side on hold, and work on the right hand side of the proposition. We can apply the formal definition of scalar multiplication to c(dA):

(c(dA))_{ij}=c(dA)_{ij}=c(da_{ij})

Since both of our simplified expressions are expressions containing only real numbers, we can apply the associative property of real multiplication to both expressions:

(cd)a_{ij}=c(da_{ij})=cda_{ij}

Since the entries of ((cd)A) equal the entries of (c(dA)), or LHS=RHS, and under the assumption that they are the same size, we can conclude that the matrices are equal from the definition of matrix equality.

Matrix Multiplication

If matrix A located on the left has the same number of columns as the number of rows in matrix B located on the right, then we can multiply A by B. To be mathematically precise, if A is  m \times n and B is n \times p, then the matrix product AB exists and is an m \times p matrix. This matrix product is defined by each of its entries, (AB)_{ij} , which is the Dot Product of the ith row of A and the jth column of B. Because of the requirement on the dimensions of the matrices, these two vectors have the same size, so the dot products make sense.

The following is an example of matrix multiplication:

A = \begin{bmatrix}
               a_{11} & a_{12} & a_{13}\\
               a_{21} & a_{22} & a_{23}\\
               \end{bmatrix} \text {and }
          B= \begin{bmatrix}
               b_{11} & b_{12}\\
               b_{21} & b_{22}\\
               b_{31} & b_{32} \\
               \end{bmatrix}

Now A has size 2x3 and B has size 3x2, so the product AB will have size 2x2. The entries of AB are the dot products of the two rows of A and the two columns of B. Slicing the A matrix in terms of rows and the B matrix in terms of columns like so:


\begin{array}{c c} &
\begin{array}{c c c}\\
\end{array} 
\\
\begin{array}{c c}
A_{1} \\
A_{2}\\
\end{array} 
&
\left[
\begin{array}{c c c}
a_{11} & a_{12} & a_{13} \\
a_{21} & a_{22} & a_{23} \\ 
\end{array}
\right] 
\end{array}

\begin{array}{c c} &
\begin{array}{c c} B_{1} & B_{2}\\
\end{array} 
\\
&
\left[
\begin{array}{c c c}
b_{11} & b_{12} \\
b_{21} & b_{22} \\
b_{31} & b_{32}\\ 
\end{array}
\right] 
\end{array}

We get product AB equal to:

=\begin{bmatrix}
    A_{1} \cdot B_{1} & A_{1} \cdot B_{2} \\
    A_{2} \cdot B_{1} & A_{2} \cdot B_{2} \\
   \end{bmatrix}


But numbers make more sense, so let’s consider two more specific matrices.

A = \begin{bmatrix}
             1& 3 & -4\\
             0 & 2 & -0.5
            \end{bmatrix} \text {and }
        B = \begin{bmatrix}
             2 & -1 \\
             5 & 0\\
             -1 & 3
            \end{bmatrix}
\begin{align}
        A_{1} \cdot B_{1} &= \langle 1,3,-4 \rangle  \cdot \langle 2,5,-1 \rangle = 2(1) + 3(5) + -4(-1) = 21 \\
        A_{1} \cdot B_{2} &= \langle 1,3,-4 \rangle  \cdot \langle -1,0,3 \rangle = 1(-1) + 3(0) + -4(3) = -13 \\
        A_{2} \cdot B_{1} &= \langle 0,2,-0.5 \rangle  \cdot \langle 2,5,-1 \rangle = 0(2) + 2(5) + (-0.5)(-1) = 10.5 \\
        A_{2} \cdot B_{2} &= \langle 0,2,-0.5 \rangle  \cdot \langle -1,0,3 \rangle = 0(-1) + 2(0) + (-0.5)(3) = -1.5 \\
        \end{align}

so the final product is

 AB = \begin{bmatrix}
               21 & -13 \\
               10.5 & -1.5
              \end{bmatrix}

This animation illustrates the process of matrix multiplication:


There are two special matrices that may appear when making basic operations. The zero matrix is a square matrix whose entries are all zero. If we call this Z, then clearly Z+A = A+Z = A for all matrices A for which this makes sense, so the zero matrix behaves like the number 0 for addition. The identity matrix is a square matrix that has aii=1, but aij=0 if i≠j. That is, the identity matrix for the 3 \times 3 case is:
 I = \begin{bmatrix}
              1 & 0 & 0 \\
              0 & 1 & 0 \\
              0 & 0 & 1
             \end{bmatrix}
The identity matrix behaves like the number 1 for real multiplication, that is AI = IA = A for all square matrices A of the right dimensions. Try practicing matrix multiplication by proving this property.
Matrix multiplication has one other interesting property: it is not commutative. Generally, ABBA for matrices. You can quickly see this for the matrices A and B above: while AB is a 2x2 matrix, BA is a 3x3 matrix.

Properties of Matrix Muliplication

Let A be m \times n and B be  n \times p Then the product AB is an m \times p matrix where:

 (AB)_{ij}= \sum_{k=1}^n{A_{ik}B_{kj}}

The above summation formula is a reasonable representation of matrix multiplication because the sum is the dot product of A_{i} and B_{j} for any matrices A and B, if both are of multiplicable size.


For matrices A, m x n, B, n x p, and C, p x q:

  1. Matrix Multiplication is Associative:(AB)C=A(BC)
  2. Matrix Multiplication is Distributive over Addition :A(B+C)=AB+AC
  3. Identity for Multiplication:AI=IA=A

Proofs of Matrix Multiplication Properties

1. Proof that Matrix Multiplication is Associative: (AB)C=A(BC)

By the definition of matrix equality, we want to show that for multiplicable matrices: A, "m x n", B "n x p", and C "p x q", their matrix product can be grouped in any manner so that entries of one grouped matrix product equal the corresponding entries of another differently grouped matrix product.

Claim: ((AB)C)_{ij}=(A(BC))_{ij}

Working from the left hand side of the proposition, we apply the definition of matrix multiplication to the matrices, (AB) and C:

((AB)C)_{ij} = \sum_{k=1}^n(AB)_{ik}C_{kj}

By the definition of matrix multiplication, we break down the matrices (AB) and C to the real number entries of matrices A, B, and C:

\sum_{k=1}^n(AB)_{ik}C_{kj} = \sum_{k=1}^n\left(\sum_{p=1}^lA_{ip}B_{pk}\right)C_{kj}

Because we have reduced the matrices down to their real number entries, properties of real numbers apply. Switching the order of summation by the commutative and associative properties of real numbers under multiplication:

\sum_{k=1}^n\left(\sum_{p=1}^lA_{ip}B_{pk}\right)C_{kj} =\sum_{p=1}^lA_{ip}\left(\sum_{k=1}^nB_{pk}C_{kj}\right)

By the definition of matrix multiplication applied to the matrix BC:

\sum_{p=1}^lA_{ip}\left(\sum_{k=1}^nB_{pk}C_{kj}\right) = \sum_{p=1}^lA_{ip}(BC)_{pj}

By the definition of matrix multiplication, we group the matrix BC with the constant A_{ip} , to form the composite matrix product:

\sum_{p=1}^lA_{ip}(BC)_{pj} =(A(BC))_{ij}

Thus:

 ((AB)C)_{ij}=(A(BC))_{ij} As Desired; matrix multiplication is associative.


2. Proof that Matrix Multiplication Over Addition is Distributive: A(B+C)=AB+AC

By the definition of matrix equality, we want to show that for any matrices where A is m x n, B is n x p, and C is p x q, the entries of the matrix sum multiplied by a matrix are equivalent to the corresponding entries for the sum of the products where a matrix is multiplied with each component matrix of the matrix sum

Claim: (A(B+C))_{ij} = (AB+AC)_{ij}

Working from the left hand side of the proposition, we apply the definition of matrix multiplication to the composite matrix (A(B+C)):

(A(B+C))_{ij}=\sum_{k=1}^nA_{ik}(B+C)_{kj}

By definition of matrix addition, we break down the composite matrix (A(B+C)) down to the real number entries of matrices A, B, and C:

\sum_{k=1}^nA_{ik}(B+C)_{kj}=\sum_{k=1}^nA_{ik}(B_{kj}+C_{kj})

Because we are now dealing with real number entries, the properties of real numbers apply and by the distributive property applied to the entries of matrix (A(B+C)):

\sum_{k=1}^nA_{ik}(B_{kj}+C_{kj})=\sum_{k=1}^n(A_{ik}B_{kj} + A_{ik}C_{kj})

We have reduced the left hand side of the proposition as much as possible.

Now, we will be working on the right hand side of the proposition. By the definition of matrix addition, we break down the composite matrix (AB+AC) into the separate matrices (AB) and (AC):

(AB+AC)_{ij}= (AB)_{ij} + (AC)_{ij}

By the definition of matrix multiplication, we break down the matrices (AB) and (AC) to the real number entries of matrices A, B, and C:

(AB)_{ij} + (AC)_{ij} =\sum_{k=1}^nA_{ik}B_{kj} + \sum_{k=1}^nA_{ik}C_{kj}

Now that we are dealing with real number entries, properties of real numbers apply. So by the associativity and commutativity of real numbers applied to the entries from matrices (AB) and (AC), we obtain:

\sum_{k=1}^nA_{ik}B_{kj} + \sum_{k=1}^nA_{ik}C_{kj}=\sum_{k=1}^n(A_{ik}B_{kj} + A_{ik}C_{kj})
Finally we have shown that the left hand side equals the right hand side of our original claim. Thus, matrix multiplication over addition is distributive.

Matrix Transposition

Another matrix operation you might see frequently is transposition. Matrix transposition replaces each row of a matrix with the corresponding column of the same matrix. The transpose is written as AT and is defined by (A^T)_{ij} = A_{ji}. In other words, if A is m x n then AT is n x m; A doesn't necessarily have to be a square matrix in order for it to have a transpose. To make all this a little clearer, let's look at the example for a non-square matrix A:

 A = \begin{bmatrix}
              1 & 6 & -4 \\
              -8 &-2 & 9 \\
             \end{bmatrix} \text{ and }
       A^T = \begin{bmatrix}
              1 & -8  \\
              6 & -2 \\
              -4 & 9 \\
             \end{bmatrix}

If we take the transpose of the above A^T , we see that:

 (A^T)^T = \begin{bmatrix}
                              1 & 6 & -4 \\
                              -8& -2& 9\\
                              \end{bmatrix}

We are back to the original matrix, A, which we started with. This leads us to one of the properties of Matrix Transposition, defined formally under #1 in the Properties below. Think intuitively about the property #1 and how it is analogous to reflecting over the main diagonal through the three a entries as shown in the picture below.

Image:Gif.jpg

Properties of Matrix Transposition

  1. Self-inverse of the transpose(A^T)^T=A
  2. Transposes preserve addition (A+B)^T=A^T+B^T
  3. Transposes reverse the ordering of the matrices (AB)^T=B^TA^T
  4. Transposes preserve scalar multiplication (kA)^T=kA^T, where  k is a scalar multiple.

Proofs of Matrix Transposition Properties

1. Proof that (A^T)^T=A

By the definition of matrix equality, we want to show that the entries for the transpose of the transpose of matrix A equals the corresponding entries of the original matrix A:

Claim: ((A^T)^T)_{ij}= A_{ij}

Working from the left hand side of the proposition, we apply the definition of matrix transpose to  A^T :

 ((A^T)^T)_{ij}= (A^T)_{ji}

Then, applying the definition of the matrix transpose to the above result:

(A^T)_{ji}  = A_{ij}

We have manipulated the left hand side so that it equals the right hand side, the corresponding entries of original matrix A. The notion of matrix equality holds so the proof is complete.

Even without the proof, this transpose property makes intuitive sense. Imagine taking the transpose twice as flipping a diagonal twice, which takes you back where you started.


2. Proof that (A+B)^T=A^T+B^T

By the definition of matrix equality, we want to show that the entries of the transpose of the matrix (A+B) are equivalent to the corresponding entries for the sum of the individual transposes of A and B.

Claim: ((A+B)^T)_{ij}= (A^T + B^T)_{ij}

Working from the left hand side of the proposition, we apply the definition of matrix transposition to the summed matrix (A+B):

 ((A+B)^T)_{ij}= (A+B)_{ji}

By the definition of matrix addition, we further reduce the matrix (A+B) down to its real number entries:

 (A+B)_{ji}= A_{ji} + B_{ji}

We have reduced the left hand side of the proposition as much as possible.


Now, working from the right hand side of the proposition, we apply the definition of matrix addition to the individual transposed matrices, A^T, and B^T:

 (A^T + B^T)_{ij} = (A^T)_{ij} + (B^T)_{ij}

By the definition of the matrix transpose applied to matrices A and B, we arrive at the same reduction that we obtained while working from the left hand side.

(A^T)_{ij} + (B^T)_{ij}  = A_{ji} + B_{ji}

Because LHS = RHS , the notion of matrix equality holds. The proof is complete.


3. Proof that  (AB)^T= B^TA^T

By the definition of matrix equality, we want to show that the entries for the transpose of the product of two matrices equal the corresponding entries of the product of separately transposed matrices in reverse order.

Claim: ((AB)^T)_{ij}=(B^TA^T)_{ij}

Starting from the left hand side of the proposition, we apply the definition of the matrix transpose to the matrix, (AB)

((AB)^T)_{ij} = (AB)_{ji}

By the definition of matrix multiplication applied to the matrix (AB):

 (AB)_{ji}=\sum_{k=1}^nA_{jk}B_{ki}

We have reduced the left hand side of the proposition as much as possible, down to the real numbered entries of matrix (AB).

Now working from the right hand side of the proposition, we apply the definition of matrix multiplication to the matrices B^T and A^T:

 (B^TA^T)_{ij}= \sum_{k=1}^n(B^T)_{ik}(A^T)_{kj}

By the definition of matrix transpose applied to matrices B and A:

\sum_{k=1}^n(B^T)_{ik}(A^T)_{kj} =\sum_{k=1}^n B_{ki}A_{jk}

Now we have reduced the matrices of B and A to their real number entries. By the commutative property of real numbers under multiplication:

\sum_{k=1}^n B_{ki}A_{jk}  =\sum_{k=1}^n A_{jk}B_{ki}

Now, we have shown that the left hand side equals the right hand side of our original claim. The notion of matrix equality holds. Thus, transposes reverse the ordering of the matrices.


4. Proof that (kA)^T=kA^T

By the definition of matrix equality, we want to show that the entries for the transpose of a scalar matrix product equal the corresponding entries for the product of a scalar and the matrix transpose.

 Claim: ((kA)^T)_{ij}=(kA^T)_{ij}

Working from the left hand side of the proposition, we apply the definition of the matrix transpose to matrix (kA):

 ((kA)^T)_{ij}= (kA)_{ji}

By the properties of multiplying a scalar with a matrix, we obtain:

(kA)_{ji}=k A_{ji}

We have reduced the left hand side as much as possible. Now moving onto the right hand side of the claim.

By the properties of multiplying a scalar with a matrix applied to the entries of the matrix (kA^T):

(kA^T)_{ij}= k (A^T)_{ij}

Then, by the definition of matrix transpose applied to matrix A:

k (A^T)_{ij}=k A_{ji}
We have shown that the reduction of the left hand side equals the right hand side and vice versa. The notion of matrix equality holds. Thus, transposes preserve scalar multiplication.

Matrices Are Functions

One of the most common ways to work with matrices is that matrices can represent functions from one Cartesian space to another. We can see this if we think of a point as a vector, and then of a vector as a matrix. So we can write the vector  V = \langle 3,-2,5 \rangle as  V = \begin{bmatrix} 3 \\ -2 \\ 5 \end{bmatrix} , and in this form it is a 3x1 matrix. Then we can multiply a 3x3 matrix by V and get another 3x1 matrix as a result. For example,

if   \begin{bmatrix}
              1 & 6 & -4 \\
              -8 & -2 & 9 \\
              5 & 7 & 3
             \end{bmatrix} and  V = \begin{bmatrix} 3 \\ -2 \\ 5 \end{bmatrix} , then  AV = \begin{bmatrix}
             3 -12 -20 \\
             -24 + 4 + 45 \\
             15 - 14 + 15
            \end{bmatrix} 
= \begin{bmatrix} -29 \\ 25 \\ 16 \end{bmatrix}

So the matrix A represents a function on 3D Cartesian space. Matrix multiplication has the properties we would expect a set of functions to have: they are associative, are linear (that is, if V = V1 + V2, then AV = AV1 + AV2), and has an identity I. In the later pages on transformations we will see exactly how we use matrices as functions in computer graphics. One such page is Math for Computer Graphics and Computer Vision.

References

This page was originally written by Steve Cunningham.

ch_data_struct4.gif (image for three dimensional matrix)

Source for transpose figure: http://www.katjaas.nl/transpose/transpose.html

Additional Information

Another explanation of matrix operations: http://www.miislita.com/information-retrieval-tutorial/matrix-tutorial-2-matrix-operations.html

http://www.millersville.edu/~bikenaga/linear-algebra/matrix-properties/matrix-properties.html

Personal tools