# Matrix

(Difference between revisions)
 Revision as of 11:36, 26 July 2013 (edit) (→Matrix Addition)← Previous diff Revision as of 15:31, 26 July 2013 (edit) (undo) (→Matrix Multiplication)Next diff → Line 187: Line 187: ===Matrix Multiplication=== ===Matrix Multiplication=== - If matrix ''A'' (located on the left) has the same number of columns as the number of rows in matrix ''B'' (located on the right), then we can multiply the matrices. To be mathematically precise, if ''A'' is $m \times n$ and ''B'' is $n \times p$, then the matrix product ''AB'' exists and is a $m \times p$ matrix. This matrix product is defined by defining each of its entries: $(AB)_{ij}$ as the [[Dot Product]] of the ith row of ''A'' and the jth column of ''B''. Because of the requirement on the dimensions of the matrices, these two vectors have the same size, so the dot products make sense. + If matrix ''A'' located on the left has the same number of columns as the number of rows in matrix ''B'' located on the right, then we can multiply ''A'' by ''B.'' To be mathematically precise, if ''A'' is $m \times n$ and ''B'' is $n \times p$, then the matrix product ''AB'' exists and is an $m \times p$ matrix. This matrix product is defined by each of its entries, $(AB)_{ij}$, which is the [[Dot Product]] of the ith row of ''A'' and the jth column of ''B''. Because of the requirement on the dimensions of the matrices, these two vectors have the same size, so the dot products make sense. The following is an example of matrix multiplication: The following is an example of matrix multiplication: Line 236: Line 236: [/itex] [/itex] - We get that the product ''AB'' is equal to: + We get product ''AB'' equal to: ::$=\begin{bmatrix} ::[itex]=\begin{bmatrix} A_{1} \cdot B_{1} & A_{1} \cdot B_{2} \\ A_{1} \cdot B_{1} & A_{1} \cdot B_{2} \\ Line 244: Line 244: - But numbers make even more sense, so let’s consider the two matrices again. + But numbers make more sense, so let’s consider two more specific matrices. ::[itex]A = \begin{bmatrix} ::[itex]A = \begin{bmatrix} 1& 3 & -4\\ 1& 3 & -4\\ Line 285: Line 285: Matrix multiplication has one other interesting property: it is not commutative. Generally, ''AB'' ≠ ''BA'' for matrices. You can quickly see this for the matrices ''A'' and ''B'' above: while ''AB'' is a 2x2 matrix, ''BA'' is a 3x3 matrix. Matrix multiplication has one other interesting property: it is not commutative. Generally, ''AB'' ≠ ''BA'' for matrices. You can quickly see this for the matrices ''A'' and ''B'' above: while ''AB'' is a 2x2 matrix, ''BA'' is a 3x3 matrix. ====Properties of Matrix Muliplication==== ====Properties of Matrix Muliplication==== - Let ''A'' be [itex]m \times n$ and ''B'' be $n \times p$ Then the product ''AB'' is a $m \times p$ matrix where: + Let ''A'' be $m \times n$ and ''B'' be $n \times p$ Then the product ''AB'' is an $m \times p$ matrix where: :::$(AB)_{ij}= \sum_{k=1}^n{A_{ik}B_{kj}} :::[itex] (AB)_{ij}= \sum_{k=1}^n{A_{ik}B_{kj}}$ [/itex] The above summation formula is a reasonable representation of matrix multiplication because the sum is the dot product of $A_{i}$ and $B_{j}$ for any matrices ''A'' and ''B'', if both are of multiplicable size. The above summation formula is a reasonable representation of matrix multiplication because the sum is the dot product of $A_{i}$ and $B_{j}$ for any matrices ''A'' and ''B'', if both are of multiplicable size. + + For matrices A, ''m x n,'' B, ''n x p,'' and C, ''p x q:'' #'''Matrix Multiplication is Associative''':$(AB)C=A(BC)$ #'''Matrix Multiplication is Associative''':$(AB)C=A(BC)$ #'''Matrix Multiplication is Distributive over Addition ''':$A(B+C)=AB+AC$ #'''Matrix Multiplication is Distributive over Addition ''':$A(B+C)=AB+AC$ Line 298: Line 300: 1. '''Proof that Matrix Multiplication is Associative:''' $(AB)C=A(BC)$ 1. '''Proof that Matrix Multiplication is Associative:''' $(AB)C=A(BC)$ - By the definition of matrix equality, we want to show (WTS) that for multiplicable matrices: A of "m x n" size, B of "n x p" size, and C of "p x q" size, their matrix product can be grouped in any manner so that corresponding entries of one grouped matrix product equal the corresponding entries of another differently grouped matrix product. + By the definition of matrix equality, we want to show that for multiplicable matrices: A, "m x n", B "n x p", and C "p x q", their matrix product can be grouped in any manner so that entries of one grouped matrix product equal the corresponding entries of another differently grouped matrix product. - :::::$WTS: [(AB)C]_{ij}=[A(BC)]_{ij}$ + :::::$Claim: ((AB)C)_{ij}=(A(BC))_{ij}$ - Working from the left hand side (LHS) of the proposition, we apply the definition of matrix multiplication to the matrices, (AB) and C: + Working from the left hand side of the proposition, we apply the definition of matrix multiplication to the matrices, (AB) and C: - :::::$[(AB)C]_{ij} = \sum_{k=1}^n(AB)_{ik}C_{kj}$ + :::::$((AB)C)_{ij} = \sum_{k=1}^n(AB)_{ik}C_{kj}$ - By the definition of matrix multiplication, we break down the matrices (AB) and C to their respective real number entries: + By the definition of matrix multiplication, we break down the matrices (AB) and C to the real number entries of matrices A, B, and C: - :::::$\sum_{k=1}^n(AB)_{ik}C_{kj} = \sum_{k=1}^n\left(\sum_{p=1}^l(a_{ip}b_{pk})\right)c_{kj}$ + :::::$\sum_{k=1}^n(AB)_{ik}C_{kj} = \sum_{k=1}^n\left(\sum_{p=1}^lA_{ip}B_{pk}\right)C_{kj}$ Because we have reduced the matrices down to their real number entries, properties of real numbers apply. Switching the order of summation by the commutative and associative properties of real numbers under multiplication: Because we have reduced the matrices down to their real number entries, properties of real numbers apply. Switching the order of summation by the commutative and associative properties of real numbers under multiplication: - :::::$\sum_{k=1}^n\left(\sum_{p=1}^l(a_{ip}b_{pk})\right)c_{kj} =\sum_{p=1}^la_{ip}\left(\sum_{k=1}^n(b_{pk}c_{kj})\right)$ + :::::$\sum_{k=1}^n\left(\sum_{p=1}^lA_{ip}B_{pk}\right)C_{kj} =\sum_{p=1}^lA_{ip}\left(\sum_{k=1}^nB_{pk}C_{kj}\right)$ By the definition of matrix multiplication applied to the matrix BC: By the definition of matrix multiplication applied to the matrix BC: - :::::$\sum_{p=1}^la_{ip}\left(\sum_{k=1}^n(b_{pk}c_{kj})\right) = \sum_{p=1}^la_{ip}(BC)_{pj}$ + :::::$\sum_{p=1}^lA_{ip}\left(\sum_{k=1}^nB_{pk}C_{kj}\right) = \sum_{p=1}^lA_{ip}(BC)_{pj}$ - By the definition of matrix multiplication, we group the matrix BC with the constant $a_{ip}$, to form the composite matrix product: + By the definition of matrix multiplication, we group the matrix BC with the constant $A_{ip}$, to form the composite matrix product: - :::::$\sum_{p=1}^la_{ip}(BC)_{pj} =[A(BC)]_{ij}$ + :::::$\sum_{p=1}^lA_{ip}(BC)_{pj} =(A(BC))_{ij}$ Thus: Thus: - :::::$[(AB)C]_{ij}=[A(BC)]_{ij}$ As Desired. Proof is complete. + :::::$((AB)C)_{ij}=(A(BC))_{ij}$ As Desired; matrix multiplication is associative. Line 330: Line 332: 2. '''Proof that Matrix Multiplication Over Addition is Distributive:''' $A(B+C)=AB+AC$ 2. '''Proof that Matrix Multiplication Over Addition is Distributive:''' $A(B+C)=AB+AC$ - By the definition of matrix equality, we want to show (WTS) that for any matrices where A is ''m x n,'' B is ''n x p,'' and C is ''p x q,'' the corresponding entries of the matrix sum multiplied by a matrix is equivalent to the corresponding entries for the sum of the products where a matrix is multiplied with each component matrix of the matrix sum + By the definition of matrix equality, we want to show that for any matrices where A is ''m x n,'' B is ''n x p,'' and C is ''p x q,'' the entries of the matrix sum multiplied by a matrix are equivalent to the corresponding entries for the sum of the products where a matrix is multiplied with each component matrix of the matrix sum - :::::$WTS: [A(B+C)]_{ij} = [AB+AC]_{ij}$ + :::::$Claim: (A(B+C))_{ij} = (AB+AC)_{ij}$ - Working from the left hand side (LHS) of the proposition, we apply the definition of matrix multiplication to the composite matrix [A(B+C)]: + Working from the left hand side of the proposition, we apply the definition of matrix multiplication to the composite matrix (A(B+C)): - :::::$[A(B+C)]_{ij}=\sum_{k=1}^nA_{ik}(B+C)_{kj}$ + :::::$(A(B+C))_{ij}=\sum_{k=1}^nA_{ik}(B+C)_{kj}$ - By definition of matrix addition, we break down the composite matrix [A(B+C)] down to its real number entries: + By definition of matrix addition, we break down the composite matrix (A(B+C)) down to the real number entries of matrices A, B, and C: - :::::$\sum_{k=1}^nA_{ik}(B+C)_{kj}=\sum_{k=1}^na_{ik}(b_{kj}+c_{kj})$ + :::::$\sum_{k=1}^nA_{ik}(B+C)_{kj}=\sum_{k=1}^nA_{ik}(B_{kj}+C_{kj})$ - Because we are now dealing with real number entries, the properties of real numbers apply and by the distributive property applied to the entries of matrix [A(B+C)]: + Because we are now dealing with real number entries, the properties of real numbers apply and by the distributive property applied to the entries of matrix (A(B+C)): - :::::$\sum_{k=1}^na_{ik}(b_{kj}+c_{kj})=\sum_{k=1}^n(a_{ik}b_{kj} + a_{ik}c_{kj})$ + :::::$\sum_{k=1}^nA_{ik}(B_{kj}+C_{kj})=\sum_{k=1}^n(A_{ik}B_{kj} + A_{ik}C_{kj})$ - We have reduced the left hand side (LHS) of the proposition as much as possible. + We have reduced the left hand side of the proposition as much as possible. - Now, we will be working on the right hand side (RHS) of the proposition. By the definition of matrix addition, we break down the composite matrix [AB+AC] into the separate matrices of (AB) and (AC): + Now, we will be working on the right hand side of the proposition. By the definition of matrix addition, we break down the composite matrix (AB+AC) into the separate matrices (AB) and (AC): - :::::$[AB+AC]_{ij}= (AB)_{ij} + (AC)_{ij}$ + :::::$(AB+AC)_{ij}= (AB)_{ij} + (AC)_{ij}$ - By the definition of matrix multiplication, we break down the matrices (AB) and (AC) to their real number entries: + By the definition of matrix multiplication, we break down the matrices (AB) and (AC) to the real number entries of matrices A, B, and C: - :::::$(AB)_{ij} + (AC)_{ij} =\sum_{k=1}^na_{ik}b_{kj} + \sum_{k=1}^na_{ik}c_{kj}$ + :::::$(AB)_{ij} + (AC)_{ij} =\sum_{k=1}^nA_{ik}B_{kj} + \sum_{k=1}^nA_{ik}C_{kj}$ - Now that we are dealing with real number entries, properties of real numbers apply. So by the associativity and commutativity of real numbers applied to the real number entries from matrices (AB) and (AC), we obtain: + Now that we are dealing with real number entries, properties of real numbers apply. So by the associativity and commutativity of real numbers applied to the entries from matrices (AB) and (AC), we obtain: - :::::$\sum_{k=1}^na_{ik}b_{kj} + \sum_{k=1}^na_{ik}c_{kj}=\sum_{k=1}^n(a_{ik}b_{kj} + a_{ik}c_{kj})$ + :::::$\sum_{k=1}^nA_{ik}B_{kj} + \sum_{k=1}^nA_{ik}C_{kj}=\sum_{k=1}^n(A_{ik}B_{kj} + A_{ik}C_{kj})$ - Finally: + Finally we have shown that the left hand side equals the right hand side of our original claim. Thus, matrix multiplication over addition is distributive. - :::::$LHS=RHS$ So proof is complete. + }} }}

## Revision as of 15:31, 26 July 2013

This is a Helper Page for:
Blue Fern
Summation Notation
Change of Coordinate Systems
Math for Computer Graphics and Computer Vision

A matrix is a rectangular array of numbers that can be used to store numbers for later access. In this helper page, we will discuss the mathematical properties of matrices.

## Size

A matrix is typically described in terms of its size. If a Matrix M has m rows and n columns, we say that M has size m x n, or more simply, we say M is m x n. The numbers m and n are sometimes called the dimensions of M.

## Matrix Notation

Matrices and their entries are closely related. To avoid confusion, separate notation exists for both matrices and entries.

Before we begin, we must introduce the general notation used to indicate a location of an entry in a matrix. For a matrix A, $A_{ij}$ indicates the entry located in the ith row and jth column. Below is matrix A with entries whose row and column locations are explicitly shown.

$\begin{pmatrix} a_{11} & a_{12}& \cdots & a_{1j}\\ a_{21} & a_{22}& \cdots & a_{2j}\\ \vdots & \vdots & \ddots & \vdots \\ a_{i1} & a_{i2} & \cdots & a_{ij} \end{pmatrix}$

1.Notation for a Matrix: $A=[a_{ij}]$

In other words, a matrix can be defined by just a capital letter or by the genral entry inside brackets.
• A matrix is the same thing as its general entry enclosed in brackets.

2.Notation for an Entry: $a_{ij}=(A)_{ij}$

In other words, an entry, as typically represented as a lowercase letter with a $ij$ subscript, can also be written by enclosing the name of the matrix with parenthesis. This convention applies even to matrices such as C(BA) which has the typical entry denoted as $(C(BA))_{ij}$
• $A_{ij}$ and $a_{ij}$are two ways of saying the same thing.

## Matrix Equality

In order to prove identities about matrices, we need to first define what it means for two matrices to be equal!

Definition: Two matrices A and B are equal and we write A=B, if they are the same size and $a_{ij}=b_{ij}$ for all i rows and j columns.

In other words, matrices are equal if they are of the same size and have the same corresponding entries.

In proving identities and properties of matrices, this definition comes in handy because the proofs rely on proving that the general entry of one matrix equals to the general entry of the other ($a_{ij}=b_{ij}$), which would therefore mean that both matrices is equal if their sizes are equal.

## Matrix Operations

There are three fundamental matrix operations: addition, scalar multiplication, and matrix multiplication. Properties for each operation are given below along with proofs.

For matrix addition, the sum of two matrices $A = [a_{ij}]$ and $B = [b_{ij}]$ must have the same size in which case their matrix sum is defined as $A + B = [a_{ij} + b_{ij}]$. That is, each entry in the $A+B$ matrix is the sum of the corresponding entries in the separate matrices. For instance:

$\begin{bmatrix} a & b \\ c & d \\ \end{bmatrix} + \begin{bmatrix} e & f \\ g & h \\ \end{bmatrix} = \begin{bmatrix} a+e & b+f \\ c+g & d+h\\ \end{bmatrix}$

1. Addition is Commutative:$A+B=B+A$
2. Addition is Associative:$(A+B)+C=A+(B+C)$
3. Identity of Addition:$A+0=A$

#### Proofs for properties of Matrix Addition

1. Proof for Commutativity of Addition: $A+B=B+A$

By the definition of matrix equality, we want to show that the corresponding entries of A+B and B+A are the same:

$Claim: (A+B)_{ij}=(B+A)_{ij}$

We work from the left hand side of the above proposition and attempt to prove that it equals the right hand side of the proposition. So starting from the left, we apply the definition of matrix addition to A+B, obtaining:

$(A+B)_{ij}= A_{ij} + B_{ij}$

Notice that we have reduced matrices A and B down to their real number entries, so the commutativity of real numbers applies. That is:

$A_{ij}+B_{ij}=B_{ij} + A_{ij}$

Lastly, by the definition of matrix addition applied to B and A:

$B_{ij} + A_{ij}=(B+A)_{ij}$

Stringing the last few equalities together, we get that:

$(A+B)_{ij}=(B+A)_{ij}$

That is, we have shown that the left hand side of the original proposition equals its right hand side. Thus, matrix addition is commutative.

2. Proof for Associativity of Addition: $(A+B) + C = A + (B+C)$

By the definition of matrix equality, we want to show that the entries of (A+B)+ C and A+(B+C) are the same. That is:

$Claim: ((A+B) + C)_{ij} = (A + (B+C))_{ij}$

We work from the left hand side of the above proposition and attempt to prove that it equals the right hand side of the proposition. Starting from the left, we apply the definition of matrix addition to the big composite matrix ((A+B)+C), breaking it down into two separate matrices, (A+B) and C.

$((A+B)+C)_{ij}=(A+B)_{ij} + C_{ij}$

Then, we break down the matrix (A+B) into the separate matrices A and B into the real number entries of A and B by the definition of matrix addition:

$(A+B)_{ij} + C_{ij}=(A_{ij} + B_{ij})+ C_{ij}$

Because we are now dealing with real number entries, the associativity of real numbers applies to the entries of matrices A, B, and C:

$(A_{ij} + B_{ij})+ C_{ij} =A_{ij} + (B_{ij} + C_{ij})$

Notice that the right hand side of the above equality includes the sum of the $ij$th entries for each of the B and C matrices, which is actually just the $ij$th entry of B+C:

$A_{ij} + (B_{ij} + C_{ij})=A_{ij} + (B+C)_{ij}$

Lastly, by the definition of matrix addition applied to B+C and A:

$A_{ij} + (B+C)_{ij}=(A+ (B+C))_{ij}$
Referring back to the original proposition, we have proved that its LHS equals the RHS, thus completing the proof.

### Scalar Multiplication

Definition: For a $m \times n$ matrix $A = [a_{ij}]$ and a scalar value k, the scalar product kA is defined by $kA=[ka_{ij}]$, where kA is also $m \times n$.

The Entry definition is defined as: $(kA)_{ij} = ka_{ij}$

For example:

$2 \begin{bmatrix} 2 & 4 \\ 0 & -0.3 \end{bmatrix} = \begin{bmatrix} 4 & 8 \\ 0 & -0.6 \end{bmatrix}$

#### Properties of Scalar Multiplication

1. $k(A+B)=kA+kB$
2. $(cd)A=c(dA)$, where $c$ and $d$ are real numbers.

#### Proofs of Scalar Multiplication Properties

1. Proof of Property 1: $k(A+B)=kA+kB$, where A and B are the same size.

First, by the definition of matrix equality, we want to show (WTS) that the corresponding entries of the composite matrix (A+B) times a scalar is equal to the corresponding entries of the composite matrix kA+kB:

$WTS: (k(A+B))_{ij}=(kA+kB)_{ij}$

We work from the left hand side (LHS) of the above proposition and attempt to prove that it equals the right hand side (RHS) of the proposition. So starting from the LHS, we apply the formal definition of Scalar Multiplication to the composite matrix k(A+B):

$(k(A+B))_{ij}=k(A+B)_{ij}$

Now we can apply the definition of matrix addition to A and B

$k(A+B)_{ij}=k((A)_{ij}+(B)_{ij})$

Since we are left with only real numbers now ((A)_{ij} is an entry), we can apply the distributive property of real numbers

$k((A)_{ij}+(B)_{ij})=kA_{ij}+kB_{ij}$

We work from the right hand side (RHS) of the initial proposition to see if it reduces to the last simplification of the left hand side. We see that the right hand side can use some splitting up (because of the entry notation), so we use the definition of matrix addition applied to the matrix kA+kB:

$(kA+kB)_{ij}=(kA)_{ij}+(kB)_{ij}$

Now we see scalars separately attached to matrices and should immediately consider applying the definition of scalar multiplication, which should be done next to matrices kA and kB

$(kA)_{ij}+(kB)_{ij}=k(A)_(ij)+k(B)_{ij}$

Since the entries of $k(A+B)$ equal the entries of $kA+kB$, or LHS=RHS, and under the assumption that they are the same size, we can conclude that the matrices are equal from the definition of matrix equality.

2. Proof of Property 2: $((cd)A))_{ij}=(c(dA))_{ij}$

First, by the definition of matrix equality, we want to show (WTS) that the corresponding entries of the matrix A times the quantity of the scalar $c$ times $d$ is equal to the corresponding entries of a scalar $c$ times the quantity of a matrix $A$ times a scalar, $d$:

We work from the left hand side (LHS) of the above proposition and attempt to prove that it equals the right hand side (RHS) of the proposition. So starting from the LHS, we apply the formal definition of Scalar Multiplication to (cd)A:

$((cd)A)_{ij}=(cd)a_{ij}$

We can leave the left hand side on hold, and work on the right hand side of the proposition. We can apply the formal definition of scalar multiplication to c(dA):

$(c(dA))_{ij}=c(dA)_{ij}=c(da_{ij})$

Since both of our simplified expressions are expressions containing only real numbers, we can apply the associative property of real multiplication to both expressions:

$(cd)a_{ij}=c(da_{ij})=cda_{ij}$

Since the entries of $((cd)A)$ equal the entries of $(c(dA))$, or LHS=RHS, and under the assumption that they are the same size, we can conclude that the matrices are equal from the definition of matrix equality.

### Matrix Multiplication

If matrix A located on the left has the same number of columns as the number of rows in matrix B located on the right, then we can multiply A by B. To be mathematically precise, if A is $m \times n$ and B is $n \times p$, then the matrix product AB exists and is an $m \times p$ matrix. This matrix product is defined by each of its entries, $(AB)_{ij}$, which is the Dot Product of the ith row of A and the jth column of B. Because of the requirement on the dimensions of the matrices, these two vectors have the same size, so the dot products make sense.

The following is an example of matrix multiplication:

$A = \begin{bmatrix} a_{11} & a_{12} & a_{13}\\ a_{21} & a_{22} & a_{23}\\ \end{bmatrix} \text {and } B= \begin{bmatrix} b_{11} & b_{12}\\ b_{21} & b_{22}\\ b_{31} & b_{32} \\ \end{bmatrix}$

Now A has size 2x3 and B has size 3x2, so the product AB will have size 2x2. The entries of AB are the dot products of the two rows of A and the two columns of B. Slicing the A matrix in terms of rows and the B matrix in terms of columns like so:

$\begin{array}{c c} & \begin{array}{c c c}\\ \end{array} \\ \begin{array}{c c} A_{1} \\ A_{2}\\ \end{array} & \left[ \begin{array}{c c c} a_{11} & a_{12} & a_{13} \\ a_{21} & a_{22} & a_{23} \\ \end{array} \right] \end{array} \begin{array}{c c} & \begin{array}{c c} B_{1} & B_{2}\\ \end{array} \\ & \left[ \begin{array}{c c c} b_{11} & b_{12} \\ b_{21} & b_{22} \\ b_{31} & b_{32}\\ \end{array} \right] \end{array}$

We get product AB equal to:

$=\begin{bmatrix} A_{1} \cdot B_{1} & A_{1} \cdot B_{2} \\ A_{2} \cdot B_{1} & A_{2} \cdot B_{2} \\ \end{bmatrix}$

But numbers make more sense, so let’s consider two more specific matrices.

$A = \begin{bmatrix} 1& 3 & -4\\ 0 & 2 & -0.5 \end{bmatrix} \text {and } B = \begin{bmatrix} 2 & -1 \\ 5 & 0\\ -1 & 3 \end{bmatrix}$
\begin{align} A_{1} \cdot B_{1} &= \langle 1,3,-4 \rangle \cdot \langle 2,5,-1 \rangle = 2(1) + 3(5) + -4(-1) = 21 \\ A_{1} \cdot B_{2} &= \langle 1,3,-4 \rangle \cdot \langle -1,0,3 \rangle = 1(-1) + 3(0) + -4(3) = -13 \\ A_{2} \cdot B_{1} &= \langle 0,2,-0.5 \rangle \cdot \langle 2,5,-1 \rangle = 0(2) + 2(5) + (-0.5)(-1) = 10.5 \\ A_{2} \cdot B_{2} &= \langle 0,2,-0.5 \rangle \cdot \langle -1,0,3 \rangle = 0(-1) + 2(0) + (-0.5)(3) = -1.5 \\ \end{align}

so the final product is

$AB = \begin{bmatrix} 21 & -13 \\ 10.5 & -1.5 \end{bmatrix}$

This animation illustrates the process of matrix multiplication:

There are two special matrices that may appear when making basic operations. The zero matrix is a square matrix whose entries are all zero. If we call this Z, then clearly Z+A = A+Z = A for all matrices A for which this makes sense, so the zero matrix behaves like the number 0 for addition. The identity matrix is a square matrix that has aii=1, but aij=0 if i≠j. That is, the identity matrix for the $3 \times 3$ case is:
$I = \begin{bmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{bmatrix}$
The identity matrix behaves like the number 1 for real multiplication, that is $AI = IA = A$ for all square matrices A of the right dimensions. Try practicing matrix multiplication by proving this property.
Matrix multiplication has one other interesting property: it is not commutative. Generally, ABBA for matrices. You can quickly see this for the matrices A and B above: while AB is a 2x2 matrix, BA is a 3x3 matrix.

#### Properties of Matrix Muliplication

Let A be $m \times n$ and B be $n \times p$ Then the product AB is an $m \times p$ matrix where:

$(AB)_{ij}= \sum_{k=1}^n{A_{ik}B_{kj}}$

The above summation formula is a reasonable representation of matrix multiplication because the sum is the dot product of $A_{i}$ and $B_{j}$ for any matrices A and B, if both are of multiplicable size.

For matrices A, m x n, B, n x p, and C, p x q:

1. Matrix Multiplication is Associative:$(AB)C=A(BC)$
2. Matrix Multiplication is Distributive over Addition :$A(B+C)=AB+AC$
3. Identity for Multiplication:$AI=IA=A$

#### Proofs of Matrix Multiplication Properties

1. Proof that Matrix Multiplication is Associative: $(AB)C=A(BC)$

By the definition of matrix equality, we want to show that for multiplicable matrices: A, "m x n", B "n x p", and C "p x q", their matrix product can be grouped in any manner so that entries of one grouped matrix product equal the corresponding entries of another differently grouped matrix product.

$Claim: ((AB)C)_{ij}=(A(BC))_{ij}$

Working from the left hand side of the proposition, we apply the definition of matrix multiplication to the matrices, (AB) and C:

$((AB)C)_{ij} = \sum_{k=1}^n(AB)_{ik}C_{kj}$

By the definition of matrix multiplication, we break down the matrices (AB) and C to the real number entries of matrices A, B, and C:

$\sum_{k=1}^n(AB)_{ik}C_{kj} = \sum_{k=1}^n\left(\sum_{p=1}^lA_{ip}B_{pk}\right)C_{kj}$

Because we have reduced the matrices down to their real number entries, properties of real numbers apply. Switching the order of summation by the commutative and associative properties of real numbers under multiplication:

$\sum_{k=1}^n\left(\sum_{p=1}^lA_{ip}B_{pk}\right)C_{kj} =\sum_{p=1}^lA_{ip}\left(\sum_{k=1}^nB_{pk}C_{kj}\right)$

By the definition of matrix multiplication applied to the matrix BC:

$\sum_{p=1}^lA_{ip}\left(\sum_{k=1}^nB_{pk}C_{kj}\right) = \sum_{p=1}^lA_{ip}(BC)_{pj}$

By the definition of matrix multiplication, we group the matrix BC with the constant $A_{ip}$, to form the composite matrix product:

$\sum_{p=1}^lA_{ip}(BC)_{pj} =(A(BC))_{ij}$

Thus:

$((AB)C)_{ij}=(A(BC))_{ij}$ As Desired; matrix multiplication is associative.

2. Proof that Matrix Multiplication Over Addition is Distributive: $A(B+C)=AB+AC$

By the definition of matrix equality, we want to show that for any matrices where A is m x n, B is n x p, and C is p x q, the entries of the matrix sum multiplied by a matrix are equivalent to the corresponding entries for the sum of the products where a matrix is multiplied with each component matrix of the matrix sum

$Claim: (A(B+C))_{ij} = (AB+AC)_{ij}$

Working from the left hand side of the proposition, we apply the definition of matrix multiplication to the composite matrix (A(B+C)):

$(A(B+C))_{ij}=\sum_{k=1}^nA_{ik}(B+C)_{kj}$

By definition of matrix addition, we break down the composite matrix (A(B+C)) down to the real number entries of matrices A, B, and C:

$\sum_{k=1}^nA_{ik}(B+C)_{kj}=\sum_{k=1}^nA_{ik}(B_{kj}+C_{kj})$

Because we are now dealing with real number entries, the properties of real numbers apply and by the distributive property applied to the entries of matrix (A(B+C)):

$\sum_{k=1}^nA_{ik}(B_{kj}+C_{kj})=\sum_{k=1}^n(A_{ik}B_{kj} + A_{ik}C_{kj})$

We have reduced the left hand side of the proposition as much as possible.

Now, we will be working on the right hand side of the proposition. By the definition of matrix addition, we break down the composite matrix (AB+AC) into the separate matrices (AB) and (AC):

$(AB+AC)_{ij}= (AB)_{ij} + (AC)_{ij}$

By the definition of matrix multiplication, we break down the matrices (AB) and (AC) to the real number entries of matrices A, B, and C:

$(AB)_{ij} + (AC)_{ij} =\sum_{k=1}^nA_{ik}B_{kj} + \sum_{k=1}^nA_{ik}C_{kj}$

Now that we are dealing with real number entries, properties of real numbers apply. So by the associativity and commutativity of real numbers applied to the entries from matrices (AB) and (AC), we obtain:

$\sum_{k=1}^nA_{ik}B_{kj} + \sum_{k=1}^nA_{ik}C_{kj}=\sum_{k=1}^n(A_{ik}B_{kj} + A_{ik}C_{kj})$
Finally we have shown that the left hand side equals the right hand side of our original claim. Thus, matrix multiplication over addition is distributive.

### Matrix Transposition

Another matrix operation you might see frequently is transposition. Matrix transposition replaces each row of a matrix with the corresponding column of the same matrix. The transpose is written as AT and is defined by $(A^T)_{ij} = A_{ji}$. In other words, if A is m x n then AT is n x m; A doesn't necessarily have to be a square matrix in order for it to have a transpose. To make all this a little clearer, let's look at the example for a non-square matrix A:

$A = \begin{bmatrix} 1 & 6 & -4 \\ -8 &-2 & 9 \\ \end{bmatrix} \text{ and } A^T = \begin{bmatrix} 1 & -8 \\ 6 & -2 \\ -4 & 9 \\ \end{bmatrix}$

If we take the transpose of the above $A^T$, we see that:

$(A^T)^T = \begin{bmatrix} 1 & 6 & -4 \\ -8& -2& 9\\ \end{bmatrix}$

We are back to the original matrix, A, which we started with. This leads us to one of the properties of Matrix Transposition, defined formally under #1 in the Properties below. Think intuitively about the property #1 and how it is analogous to reflecting over the main diagonal through the three $a$ entries as shown in the picture below.

#### Properties of Matrix Transposition

1. Self-inverse of the transpose$(A^T)^T=A$
2. Transposes preserve addition$(A+B)^T=A^T+B^T$
3. Transposes reverse the ordering of the matrices$(AB)^T=B^TA^T$
4. Transposes preserve scalar multiplication $(kA)^T=kA^T$, where $k$ is a scalar multiple.

#### Proofs of Matrix Transposition Properties

1. Proof that $(A^T)^T=A$

By the definition of matrix equality, we want to show (WTS) that the corresponding entries for the transpose of matrix A transposed equal the corresponding entries of the original matrix A.

$WTS: [(A^T)^T]_{ij}= A_{ij}$

Working from the left hand side (LHS) of the proposition, we apply the definition of matrix transpose to $A^T$:

$[(A^T)^T]_{ij}= (A^T)_{ji}$

Then, applying the definition of the matrix transpose again, but to $A$:

$(A^T)_{ji} = A_{ij}$

We have manipulated the LHS so that it equals the RHS, the corresponding entries of original matrix A. The notion of matrix equality holds so the proof is complete.

• Note: Even without the proof, this transpose property makes intuitive sense because say, if you imagine taking the transpose twice as flipping over a diagonal twice, then you are back where you started.

2. Proof that $(A+B)^T=A^T+B^T$

By the definition of matrix equality, we want to show (WTS) that the corresponding entries for the transpose of the summed matrix (A+B) is equivalent to the corresponding entries for the sum of individually transposed matrices.

$WTS: ((A+B)^T)_{ij}= (A^T + B^T)_{ij}$

Working from the left hand side (LHS) of the proposition, we apply the definition of matrix transposition to the summed matrix (A+B):

$((A+B)^T)_{ij}= (A+B)_{ji}$

By the definition of matrix addition, we further reduce the matrix (A+B) down to its real number entries:

$(A+B)_{ji}= a_{ji} + b_{ji}$

We have reduced the left hand side of the proposition as much as possible.

Now, working from the right hand side (RHS) of the proposition, we apply the definition of matrix addition to the individual transposed matrices, $A^T$, and $B^T$:

$(A^T + B^T)_{ij} = (A^T)_{ij} + (B^T)_{ij}$

By the definition of the matrix transpose applied to matrices A and B, we reduce them down to their respective real number entries. We arrive at the same reduction that we obtained while working from the LHS.

$(A^T)_{ij} + (B^T)_{ij} = a_{ji} + b_{ji}$

Because $LHS = RHS$, the notion of matrix equality holds. The proof is complete.

3. Proof that $(AB)^T= B^TA^T$

By the definition of matrix equality, we want to show (WTS) that the corresponding entries for the transpose of the matrix product equals the corresponding entries of the product of separately transposed matrices in reverse order.

$WTS: ((AB)^T)_{ij}=(B^TA^T)_{ij}$

Starting from the left hand side (LHS) of the proposition, we apply the definition of the matrix transpose to the matrix, (AB)

$((AB)^T)_{ij} = (AB)_{ji}$

By the definition of matrix multiplication applied to the matrix (AB):

$(AB)_{ji}=\sum_{k=1}^na_{jk}b_{ki}$

We have reduced the left hand side of the proposition as much as possible, down to the real numbered entries of matrix (AB).

Now working from the right hand side (RHS) of the proposition, we apply the definition of matrix multiplication to the matrices $B^T$ and $A^T$:

$(B^TA^T)_{ij}= \sum_{k=1}^n(B^T)_{ik}(A^T)_{kj}$

By the definition of matrix transpose applied to matrices B and A:

$\sum_{k=1}^n(B^T)_{ik}(A^T)_{kj} =\sum_{k=1}^n b_{ki}a_{jk}$

Now we have reduced the matrices of B and A to their real number entries. So by the commutative property of real numbers under multiplication:

$\sum_{k=1}^n b_{ki}a_{jk} =\sum_{k=1}^n a_{jk}b_{ki}$

Now, $LHS = RHS$. The notion of matrix equality holds and the proof is complete.

4. Proof that $(kA)^T=kA^T$

By the definition of matrix equality, we want to show (WTS) that the corresponding entries for the transpose of a scalar matrix product equals the corresponding entries for the product of a scalar and the matrix transpose.

$WTS: ((kA)^T)_{ij}=(kA^T)_{ij}$

Working from the left hand side (LHS) of the proposition, we apply the definition of the matrix transpose to matrix (kA):

$((kA)^T)_{ij}= (kA)_{ji}$

We have reduced matrix (kA) down to its real number entries so by multiplication of real numbers, we obtain:

$(kA)_{ji}=k A_{ji}$

We have reduced the LHS as much as possible. Now moving onto the right hand side (RHS) of the proposition.

The right hand side of the proposition already features real number entries, so by the multiplication of real numbers applied to the entries of the matrix (kA^T):

$(kA^T)_{ij}= k (A^T)_{ij}$

Then, by the definition of matrix transpose applied to matrix A:

$k (A^T)_{ij}=k A_{ji}$
Finally, $LHS = RHS$. The notion of matrix equality holds and the proof is complete.

## Matrices Are Functions

One of the most common ways to work with matrices is that matrices can represent functions from one Cartesian space to another. We can see this if we think of a point as a vector, and then of a vector as a matrix. So we can write the vector $V = \langle 3,-2,5 \rangle$ as $V = \begin{bmatrix} 3 \\ -2 \\ 5 \end{bmatrix}$, and in this form it is a 3x1 matrix. Then we can multiply a 3x3 matrix by V and get another 3x1 matrix as a result. For example,

if $\begin{bmatrix} 1 & 6 & -4 \\ -8 & -2 & 9 \\ 5 & 7 & 3 \end{bmatrix}$ and $V = \begin{bmatrix} 3 \\ -2 \\ 5 \end{bmatrix}$, then $AV = \begin{bmatrix} 3 -12 -20 \\ -24 + 4 + 45 \\ 15 - 14 + 15 \end{bmatrix} = \begin{bmatrix} -29 \\ 25 \\ 16 \end{bmatrix}$

So the matrix A represents a function on 3D Cartesian space. Matrix multiplication has the properties we would expect a set of functions to have: they are associative, are linear (that is, if V = V1 + V2, then AV = AV1 + AV2), and has an identity I. In the later pages on transformations we will see exactly how we use matrices as functions in computer graphics. One such page is Math for Computer Graphics and Computer Vision.

## References

 (image for three dimensional matrix)


Source for transpose figure: http://www.katjaas.nl/transpose/transpose.html