A matrix is a rectangular array of numbers that can be used to store numbers for later access. In this helper page, we will discuss the mathematical properties of matrices.
A matrix is typically described in terms of its size.
If a Matrix M has m rows and n columns, we say that M has size m x n, or more simply, we say M is m x n.
The numbers m and n are sometimes called the dimensions of M.
Matrix Notation
Matrices and their entries are closely related. To avoid confusion, separate notation exists for both matrices and entries.
Before we begin, we must introduce the general notation used to indicate a location of an entry in a matrix. For a matrix A, indicates the entry located in the ith row and jth column. Below is matrix A with entries whose row and column locations are explicitly shown.
1.Notation for a Matrix:
In other words, a matrix can be defined by just a capital letter or by the genral entry inside brackets.
A matrix is the same thing as its general entry enclosed in brackets.
2.Notation for an Entry:
In other words, an entry, as typically represented as a lowercase letter with a subscript, can also be written by enclosing the name of the matrix with parenthesis. This convention applies even to matrices such as C(BA) which has the typical entry denoted as
and are two ways of saying the same thing.
Matrix Equality
In order to prove identities about matrices, we need to first define what it means for two matrices to be equal!
Definition: Two matrices A and B are equal and we write A=B, if they are the same size and for all i rows and j columns.
In other words, matrices are equal if they are of the same size and have the same corresponding entries.
In proving identities and properties of matrices, this definition comes in handy because the proofs rely on proving that the general entry of one matrix equals to the general entry of the other (), which would therefore mean that both matrices is equal if their sizes are equal.
Matrix Operations
There are three fundamental matrix operations: addition, scalar multiplication, and matrix multiplication. Properties for each operation are given below along with proofs.
Matrix Addition
For matrix addition, two matrices and must have the same size in which case their matrix sum is defined as . That is, each entry in the matrix is the sum of the corresponding entries in the separate matrices. For instance:
Properties of Matrix Addition
Addition is Commutative:
Addition is Associative:
Identity of Addition:
Proofs for properties of Matrix Addition
[show more][hide]
1. Proof for Commutativity of Addition:
By the definition of matrix equality, we want to show (WTS) that the corresponding entries of A+B and B+A are the same:
We work from the left hand side (LHS) of the above proposition and attempt to prove that it equals the right hand side (RHS) of the proposition.
So starting from the LHS, we apply the definition of matrix addition to A+B, obtaining:
Notice that we have reduced matrices A and B down to their real number entries, so the commutativity of real numbers apply. That is:
Lastly, by the definition of matrix addition applied to B and A:
Stringing the last few equalities together, we get that:
That is, we have shown that the left hand side of the original proposition equals its right hand side. The proof is complete.
2. Proof for Associativity of Addition:
By the definition of matrix equality, we want to show (WTS) that (A+B)+C and A+(B+C) are the same. That is:
We work from the left hand side (LHS) of the above proposition and attempt to prove that it equals the right hand side (RHS) of the proposition.
Starting from the LHS, we apply the definition of matrix addition to the big composite matrix [(A+B)+C], breaking it down into two separate matrices, (A+B) and C.
Then, we break down the matrix (A+B) into the separate matrices A and B into their real numbered entries by the definition of matrix addition:
Because we are now dealing with real numbered entries, the associativity of real numbers apply to the entries of matrices B and C:
Notice that the RHS of the above equality incudes the th entries for each of the B and C matrices, which is actually just the th entry of B+C:
Lastly, by the definition of matrix addition applied to B+C and A:
Referring back to the original proposition, we have proved that its LHS equals the RHS, thus completing the proof.
Scalar Multiplication
Definition: For a matrix and a scalar value k, the scalar product kA is defined by , where kA is also .
The Entry definition is defined as:
For example:
Properties of Scalar Multiplication
, where and are real numbers.
Proofs of Scalar Multiplication Properties
[show more][hide]
1. Proof of Property 1: , where A and B are the same size.
First, by the definition of matrix equality, we want to show (WTS) that the corresponding entries of the composite matrix (A+B) times a scalar is equal to the corresponding entries of the composite matrix kA+kB:
We work from the left hand side (LHS) of the above proposition and attempt to prove that it equals the right hand side (RHS) of the proposition. So starting from the LHS, we apply the formal definition of Scalar Multiplication to the composite matrix k(A+B):
Now we can apply the definition of matrix addition to A and B
Since we are left with only real numbers now ((A)_{ij} is an entry), we can apply the distributive property of real numbers
We work from the right hand side (RHS) of the initial proposition to see if it reduces to the last simplification of the left hand side. We see that the right hand side can use some splitting up (because of the entry notation), so we use the definition of matrix addition applied to the matrix kA+kB:
Now we see scalars separately attached to matrices and should immediately consider applying the definition of scalar multiplication, which should be done next to matrices kA and kB
Since the entries of equal the entries of , or LHS=RHS, and under the assumption that they are the same size, we can conclude that the matrices are equal from the definition of matrix equality.
2. Proof of Property 2:
First, by the definition of matrix equality, we want to show (WTS) that the corresponding entries of the matrix A times the quantity of the scalar times is equal to the corresponding entries of a scalar times the quantity of a matrix times a scalar, :
We work from the left hand side (LHS) of the above proposition and attempt to prove that it equals the right hand side (RHS) of the proposition. So starting from the LHS, we apply the formal definition of Scalar Multiplication to (cd)A:
We can leave the left hand side on hold, and work on the right hand side of the proposition. We can apply the formal definition of scalar multiplication to c(dA):
Since both of our simplified expressions are expressions containing only real numbers, we can apply the associative property of real multiplication to both expressions:
Since the entries of equal the entries of , or LHS=RHS, and under the assumption that they are the same size, we can conclude that the matrices are equal from the definition of matrix equality.
Matrix Multiplication
If matrix A (located on the left) has the same number of columns as the number of rows in matrix B (located on the right), then we can multiply the matrices. To be mathematically precise, if A is and B is , then the matrix product AB exists and is a matrix. This matrix product is defined by defining each of its entries: as the Dot Product of the i^{th} row of A and the j^{th} column of B. Because of the requirement on the dimensions of the matrices, these two vectors have the same size, so the dot products make sense.
The following is an example of matrix multiplication:
Now A has size 2x3 and B has size 3x2, so the product AB will have size 2x2. The entries of AB are the dot products of the two rows of A and the two columns of B.
Slicing the A matrix in terms of rows and the B matrix in terms of columns like so:
We get that the product AB is equal to:
But numbers make even more sense, so let’s consider the two matrices again.
so the final product is
This animation illustrates the process of matrix multiplication:
There are two special matrices that may appear when making basic operations. The zero matrix is a square matrix whose entries are all zero. If we call this Z, then clearly Z+A = A+Z = A for all matrices A for which this makes sense, so the zero matrix behaves like the number 0 for addition. The identity matrix is a square matrix that has a_{ii}=1, but a_{ij}=0 if i≠j. That is, the identity matrix for the case is:
The identity matrix behaves like the number 1 for real multiplication, that is for all square matrices A of the right dimensions. Try practicing matrix multiplication by proving this property.
Matrix multiplication has one other interesting property: it is not commutative. Generally, AB ≠ BA for matrices. You can quickly see this for the matrices A and B above: while AB is a 2x2 matrix, BA is a 3x3 matrix.
Properties of Matrix Muliplication
Let A be and B be Then the product AB is a matrix where:
The above summation formula is a reasonable representation of matrix multiplication because the sum is the dot product of and for any matrices A and B, if both are of multiplicable size.
Matrix Multiplication is Associative:
Matrix Multiplication is Distributive over Addition :
Identity for Multiplication:
Proofs of Matrix Multiplication Properties
[show more][hide]
1. Proof that Matrix Multiplication is Associative:
By the definition of matrix equality, we want to show (WTS) that for multiplicable matrices: A of "m x n" size, B of "n x p" size, and C of "p x q" size, their matrix product can be grouped in any manner so that corresponding entries of one grouped matrix product equal the corresponding entries of another differently grouped matrix product.
Working from the left hand side (LHS) of the proposition, we apply the definition of matrix multiplication to the matrices, (AB) and C:
By the definition of matrix multiplication, we break down the matrices (AB) and C to their respective real number entries:
Because we have reduced the matrices down to their real number entries, properties of real numbers apply. Switching the order of summation by the commutative and associative properties of real numbers under multiplication:
By the definition of matrix multiplication applied to the matrix BC:
By the definition of matrix multiplication, we group the matrix BC with the constant , to form the composite matrix product:
Thus:
As Desired. Proof is complete.
2. Proof that Matrix Multiplication Over Addition is Distributive:
By the definition of matrix equality, we want to show (WTS) that for any matrices where A is m x n, B is n x p, and C is p x q, the corresponding entries of the matrix sum multiplied by a matrix is equivalent to the corresponding entries for the sum of the products where a matrix is multiplied with each component matrix of the matrix sum
Working from the left hand side (LHS) of the proposition, we apply the definition of matrix multiplication to the composite matrix [A(B+C)]:
By definition of matrix addition, we break down the composite matrix [A(B+C)] down to its real number entries:
Because we are now dealing with real number entries, the properties of real numbers apply and by the distributive property applied to the entries of matrix [A(B+C)]:
We have reduced the left hand side (LHS) of the proposition as much as possible.
Now, we will be working on the right hand side (RHS) of the proposition. By the definition of matrix addition, we break down the composite matrix [AB+AC] into the separate matrices of (AB) and (AC):
By the definition of matrix multiplication, we break down the matrices (AB) and (AC) to their real number entries:
Now that we are dealing with real number entries, properties of real numbers apply. So by the associativity and commutativity of real numbers applied to the real number entries from matrices (AB) and (AC), we obtain:
Finally:
So proof is complete.
Matrix Transposition
Another matrix operation you might see frequently is transposition. Matrix transposition replaces each row of a matrix with the corresponding column of the same matrix. The transpose is written as A^{T} and is defined by . In other words, if A is m x n then A^{T} is n x m;A doesn't necessarily have to be a square matrix in order for it to have a transpose. To make all this a little clearer, let's look at the example for a non-square matrix A:
If we take the transpose of the above , we see that:
We are back to the original matrix, A, which we started with. This leads us to one of the properties of Matrix Transposition, defined formally under #1 in the Properties below. Think intuitively about the property #1 and how it is analogous to reflecting over the main diagonal through the three entries as shown in the picture below.
Properties of Matrix Transposition
Self-inverse of the transpose
Transposes preserve addition
Transposes reverse the ordering of the matrices
Transposes preserve scalar multiplication , where is a scalar multiple.
Proofs of Matrix Transposition Properties
[show more][hide]
1. Proof that
By the definition of matrix equality, we want to show (WTS) that the corresponding entries for the transpose of matrix A transposed equal the corresponding entries of the original matrix A.
Working from the left hand side (LHS) of the proposition, we apply the definition of matrix transpose to :
Then, applying the definition of the matrix transpose again, but to :
We have manipulated the LHS so that it equals the RHS, the corresponding entries of original matrix A. The notion of matrix equality holds so the proof is complete.
Note: Even without the proof, this transpose property makes intuitive sense because say, if you imagine taking the transpose twice as flipping over a diagonal twice, then you are back where you started.
2. Proof that
By the definition of matrix equality, we want to show (WTS) that the corresponding entries for the transpose of the summed matrix (A+B) is equivalent to the corresponding entries for the sum of individually transposed matrices.
Working from the left hand side (LHS) of the proposition, we apply the definition of matrix transposition to the summed matrix (A+B):
By the definition of matrix addition, we further reduce the matrix (A+B) down to its real number entries:
We have reduced the left hand side of the proposition as much as possible.
Now, working from the right hand side (RHS) of the proposition, we apply the definition of matrix addition to the individual transposed matrices, , and :
By the definition of the matrix transpose applied to matrices A and B, we reduce them down to their respective real number entries. We arrive at the same reduction that we obtained while working from the LHS.
Because , the notion of matrix equality holds. The proof is complete.
3. Proof that
By the definition of matrix equality, we want to show (WTS) that the corresponding entries for the transpose of the matrix product equals the corresponding entries of the product of separately transposed matrices in reverse order.
Starting from the left hand side (LHS) of the proposition, we apply the definition of the matrix transpose to the matrix, (AB)
By the definition of matrix multiplication applied to the matrix (AB):
We have reduced the left hand side of the proposition as much as possible, down to the real numbered entries of matrix (AB).
Now working from the right hand side (RHS) of the proposition, we apply the definition of matrix multiplication to the matrices and :
By the definition of matrix transpose applied to matrices B and A:
Now we have reduced the matrices of B and A to their real number entries. So by the commutative property of real numbers under multiplication:
Now, . The notion of matrix equality holds and the proof is complete.
4. Proof that
By the definition of matrix equality, we want to show (WTS) that the corresponding entries for the transpose of a scalar matrix product equals the corresponding entries for the product of a scalar and the matrix transpose.
Working from the left hand side (LHS) of the proposition, we apply the definition of the matrix transpose to matrix (kA):
We have reduced matrix (kA) down to its real number entries so by multiplication of real numbers, we obtain:
We have reduced the LHS as much as possible. Now moving onto the right hand side (RHS) of the proposition.
The right hand side of the proposition already features real number entries, so by the multiplication of real numbers applied to the entries of the matrix (kA^T):
Then, by the definition of matrix transpose applied to matrix A:
Finally, . The notion of matrix equality holds and the proof is complete.
Matrices Are Functions
One of the most common ways to work with matrices is that matrices can represent functions from one Cartesian space to another. We can see this if we think of a point as a vector, and then of a vector as a matrix. So we can write the vector as , and in this form it is a 3x1 matrix. Then we can multiply a 3x3 matrix by V and get another 3x1 matrix as a result. For example,
if and , then
So the matrix A represents a function on 3D Cartesian space. Matrix multiplication has the properties we would expect a set of functions to have: they are associative, are linear (that is, if V = V_{1} + V_{2}, then AV = AV_{1} + AV_{2}), and has an identity I. In the later pages on transformations we will see exactly how we use matrices as functions in computer graphics. One such page is Math for Computer Graphics and Computer Vision.
References
This page was originally written by Steve Cunningham.