\begin{equation*}
\left\{ \begin{bmatrix} 1 \\ 0 \end{bmatrix}, \begin{bmatrix} 1 \\ 1 \end{bmatrix}\right\}
\end{equation*}
is a linear independent set and
\begin{equation*}
\left\{ \begin{bmatrix} 1 \\ 2 \end{bmatrix}, \begin{bmatrix} 2 \\ 4 \end{bmatrix}\right\}
\end{equation*}
is a set of linear dependent vectors in \(\mathbb{R}^2\text{.}\)
In general, if we have two vectors
\begin{equation*}
\boldsymbol{u} = \begin{bmatrix} a \\ c\end{bmatrix}, \qquad \boldsymbol{v}= \begin{bmatrix} b\\ d \end{bmatrix}
\end{equation*}
we can determine if they are linearly dependent or independent by solving
\begin{equation*}
k_1 \boldsymbol{u} + k_2 \boldsymbol{v} = \boldsymbol{0}.
\end{equation*}
To determine linear independence, we need to solve for \(k_1\) and \(k_2\text{.}\) This can be found with the augmented matrix:
\begin{equation*}
\left[ \begin{array}{rr|r} a \amp c \amp 0 \\ b \amp d \amp 0 \end{array}\right]
\end{equation*}
Row-reducing
\begin{equation*}
-c R_1 + a R_2 \to R_2 \qquad \left[ \begin{array}{rr|r} a \amp b \amp 0 \\ 0 \amp -c b + ad \amp 0 \end{array}\right]
\end{equation*}
This system has a unique solution (of \(k_1=k_2=0\)) if \(ad-bc \neq 0\) and an infinite set of solutions if \(ad-bc =0\text{.}\) Analogous situations occur in other parts of linear algebra (like the inverse matrix) that are equivalent to this and so we name this function from matrices to the reals. For
\begin{equation}
A = \begin{bmatrix} a \amp b \\ c \amp d \end{bmatrix}\tag{2.8.1}
\end{equation}
define \(\det(A) = ad-bc\text{.}\)
The matrix inverse of a
\(2 \times 2\) matrix was found in
(2.3.1). Notice that the expression
\(ad-bc\) appeared in the formula as well.
This section expands the definition to other matrix sizes and lists other properties of the determinant.
Subsection 2.8.1 Definition of the Determinant
Before formally defining the determinant for a general matrix, there is some other needed background. We first need to understand a
permutation of a set of integers. In short a permutation is a shuffling of items. In the context of determinants, we need the items to be the first
\(n\) integers.
For example, there are six permutations of \(\{1,2,3\}\text{,}\) specifically \((1,2,3), (1,3,2), (2,1,3), (2,3,1), (3,1,2)\) and \((3,2,1)\text{.}\) Mathematically, we often think of a single permutation as a function from the set \(\{1,2,\ldots, n\}\) to itself. For example, for the permutation \((2,3,1)\text{,}\) the function \(\sigma\) is
\begin{equation*}
\sigma(1) = 2 \qquad \sigma(2) = 3 \qquad \sigma(3)=1.
\end{equation*}
It is important to know that there are
\(n!\) permutations of the set
\(\{1,2,\ldots,n\}\text{.}\) Also, any permutation can be built from a number of swaps of the trivial permutation
\((1,2,\ldots,n)\text{.}\) For example in the example above
\((2,3,1)\) can be created by starting with
\((1,2,3)\) and swapping the first two elements to get
\((2,1,3)\) and then swapping the last two elements to get
\((2,3,1).\)
Definition 2.8.1.
A permutation \(\sigma\) is even if it can be generated from the trivial permutation with an even number of swaps and odd if it can be generated with an odd number of swaps. Also the sign of the permutation, denoted \(\text{sgn}\) is
\begin{equation*}
\sgn(\sigma) = \begin{cases} 1 \amp \text{if $\sigma$ is even} \\ -1 \amp \text{if $\sigma$ is odd.} \end{cases}
\end{equation*}
Definition 2.8.2.
For an \(n \times n\) matrix, \(A\text{,}\) the determinant is
\begin{equation}
\det(A) = \sum_{\sigma} \sgn(\sigma) a_{1,\sigma(1)} a_{2,\sigma(2)} \cdots a_{n,\sigma(n)}\tag{2.8.2}
\end{equation}
where the sum is over all permutations of \(n\text{.}\)
Although this defintion works for square matrices, letโs ground ourselves a bit more before moving on. First, if we have the case of a \(1 \times 1\) matrix, then
\begin{equation*}
\det(A) = \det([a_{1,1}]) = \sum_{\sigma} a_{1,\sigma(1)} = a_{1,1}
\end{equation*}
where we have used the fact that there is one permutation of the set \(\{1\}\text{,}\) resulting in the scalar that is the only entry.
Recall that there are two permutations of \(\{1,2\}\) and that is \((1,2)\) and \((2,1)\text{,}\) so for a \(2 \times 2\) matrix, we have
\begin{equation*}
\begin{aligned}\det\left(\begin{bmatrix} a \amp b \\ c \amp d \end{bmatrix}\right) \amp = \sgn((1,2)) a_{1,\sigma(1)}a_{2,\sigma(2)} + \sgn((2,1)) a_{1,\sigma(1)}a_{2,\sigma(2)} \\ \amp = (1) a_{1,1} a_{2,2} + (-1) a_{1,2} a_{2,1} = ad-bc \end{aligned}
\end{equation*}
where recall that the 2nd term has the permutation \(\sigma(1)=2, \sigma(2)=1\text{.}\) This is identical to the \(2 \times 2\) example above.
It may seem like now that we have the definition in
Definitionย 2.8.2, we can calculate the determinant of any square matrix. The following example shows how to calculate it with a
\(3 \times 3\) matrix:
Example 2.8.4.
\begin{equation*}
A = \begin{bmatrix} 3 \amp 2 \amp -1 \\ 0 \amp 2 \amp 1 \\ 1 \amp 0 \amp -2 \end{bmatrix}
\end{equation*}
Solution.
Note that the permutations of \(\{1,2,3\}\) are those that are above and for a general \(3 \times 3\) matrix \(A\)
\begin{equation}
\begin{aligned} \det(A) \amp = a_{1,1} a_{2,2} a_{3,3} - a_{1,1}a_{2,3}a_{3,2} - a_{1,2} a_{2,1} a_{3,3} \\ \amp \qquad + a_{1,2}a_{2,3}a_{3,1} + a_{1,3} a_{2,2} a_{3,1} - a_{1,3}a_{2,1}a_{3,2} \end{aligned}\tag{2.8.4}
\end{equation}
and then evaluating it with the given element of \(A\)
\begin{equation*}
\begin{aligned} \det(A) \amp = (1)(3)(2)(-2) + (-1)(3)(1)(0) + (-1)2(0)(2) + (1)2(1)(1) \\ \amp \qquad +(1)(-1)(2)(1) + (-1)(-1)(0)(0) \\ \amp = -12 +2 -2 = -12 \end{aligned}
\end{equation*}
This example isnโt too bad. However, due to the nature of permutations, the formula in
(2.8.2) is unwieldy. For a 5 by 5 matrix, there would be
\(5!=120\) terms. Fortunately, there are other formulas available to perform the calculation. One thing to notice is using a little factoring, we can simplify the formula in
(2.8.4) and make it easier to compute. This will be done formally below.
Before presenting other computational methods of the determinant, letโs look at the properties of the determinant.
Subsection 2.8.9 Adjugate Matrices
Another matrix that is related to the inverse (as we will see below) is the
adjugate matrix. In short, this is the transpose of the cofactor matrix, which was presented in
Subsectionย 2.8.5. Letโs look at another example.
Example 2.8.32.
Find the cofactor matrix and the adjugate matrix if
\begin{equation*}
A = \begin{bmatrix} -1 \amp 2 \amp -1 \\ 4 \amp 1 \amp -1 \\ 2 \amp -2 \amp 1 \end{bmatrix}
\end{equation*}
Solution.
In this case, we find the cofactor of each of the element of \(A\text{.}\) The result is
\begin{equation*}
\cof(A) = \begin{bmatrix} -1 \amp -6 \amp -10 \\ 0 \amp 1 \amp 2 \\ -1 \amp -5 \amp -9 \end{bmatrix}
\end{equation*}
You might be wondering what the adjugate is for and that is an excellent question. First of all, it satisfies the following lemma, which we will see is quite useful.
Lemma 2.8.33.
Let \(A\) be an \(n \times n\) matrix. Then
\begin{equation*}
A \adj(A) = \det(A) I
\end{equation*}
Proof.
Let
\(C = \adj(A)\) and let
\(B = A C^{\intercal}\text{.}\) Note that
\(b_{ij}\) is the product of the
\(i\)th row of
\(A\) times the
\(j\)th column of
\(C^{\intercal}\text{,}\) or the
\(j\)th row of
\(C\text{.}\)
Consider first \(b_{ii}\text{.}\) This is
\begin{equation*}
b_{ii} = a_{i1} C_{i1} + \cdots a_{ii} C_{ii} + \cdots a_{in} C_{in}
\end{equation*}
which is
\(\det(A)\) by
Theoremย 2.8.22, the Laplace Expansion method of finding determinants.
For \(b_{ij}\) where \(i \neq j\text{,}\)
\begin{equation*}
b_{ij} = a_{i1} C_{j1} + \cdots a_{ji} C_{ji} + \cdots a_{jn} C_{jn}.
\end{equation*}
This is the determinant of a matrix where row
\(j\) has been replaced by a copy of row
\(i\text{.}\) Since this matrix has two identical rows, then by
Lemmaย 2.8.9, this determinant is 0. Thus
\(B\) is a diagonal matrix with
\(\det(A)\) on each entry on the diagonal or
\begin{equation*}
B = \det(A) I,
\end{equation*}
and the result is shown.
Before moving on, letโs do an example with the adjugate to show the results of this lemma.
Example 2.8.34.
\begin{equation*}
A = \begin{bmatrix} -1 \amp 2 \amp -1 \\ 4 \amp 1 \amp -1 \\ 2 \amp -2 \amp 1 \end{bmatrix}
\end{equation*}
to verify \(A\adj(A) = \det(A) I\)
Solution.
\begin{equation*}
C = \begin{bmatrix} -1 \amp -6 \amp -10 \\ 0 \amp 1 \amp 2 \\ -1 \amp -5 \amp -9 \end{bmatrix}
\end{equation*}
The product of these is
\begin{equation*}
AC = \begin{bmatrix} -1 \amp 0 \amp 0 \\ 0 \amp -1 \amp 0 \\ 0 \amp 0 \amp -1 \end{bmatrix}
\end{equation*}
Notice that the definition of the adjugate or the statement in
Lemmaย 2.8.33 says nothing about if
\(A\) needs to be invertible. The following example shows that if
\(A\) has determinant 0, that
\(A \adj(A)\) is the zero matrix.
Example 2.8.35.
Consider the matrix
\begin{equation*}
A = \begin{bmatrix} -2 \amp -2 \amp 0 \amp 1 \\ -1 \amp 0 \amp 1 \amp -2 \\ 2 \amp 2 \amp 0 \amp -1 \\ 0 \amp 2 \amp 0 \amp 2 \end{bmatrix}
\end{equation*}
Find
\(\adj(A)\) and show that
\(A \adj(A) = Z\text{,}\) the zero matrix.
Solution.
The cofactor matrix is
\begin{equation*}
C = \cof(A) = \begin{bmatrix} -6 \amp 4 \amp -14 \amp -4 \\ 0 \amp 0 \amp 0 \amp 0 \\ -6 \amp 4 \amp -14 \amp -4 \\ 0 \amp 0 \amp 0 \amp 0 \end{bmatrix}
\end{equation*}
and the product is the matrix of all zeros.
The result of the adjugate in
Lemmaย 2.8.33 isnโt that helpful, however the following shows a way to find the inverse.
Lemma 2.8.36.
Let \(A\) be an \(n \times n\) matrix. If \(A\) is invertible then,
\begin{equation*}
A^{-1} = \frac{1}{\det(A)} \adj(A)
\end{equation*}
Proof.
Note that \(A\) is invertible then \(\det(A)\text{,}\)
\begin{equation*}
\begin{aligned} A^{-1} \amp = \frac{1}{\det(A)} \adj(A), \qquad \qquad \text{Multiply on the left by $A$} \\ A A^{-1} \amp = A \frac{1}{\det(A)} \adj(A), \\ I \amp = \frac{1}{\det(A)} A \adj(A), \\ \det(A) I \amp = A \adj(A).
\end{aligned}
\end{equation*}
Example 2.8.37.
\begin{equation*}
A = \begin{bmatrix} 3 \amp 3 \amp -1 \\ 2 \amp 1 \amp -3 \\ 0 \amp 2 \amp 5 \end{bmatrix}
\end{equation*}
Solution.
First, the cofactor matrix is
\begin{equation*}
\cof(A) = \begin{bmatrix} 11 \amp -10 \amp 4 \\ -17 \amp 15 \amp -6 \\ -8 \amp 7 \amp -3 \end{bmatrix}
\end{equation*}
Note: this takes a lot of work (there are nine \(2 \times 2\) determinants). Next, we take the determinant of \(A\text{.}\) Again this is not shown, but \(\det(A) = -1\text{.}\) The inverse therefore is
\begin{equation*}
\begin{aligned} A^{-1} \amp = \frac{1}{\det(A)} \adj(A) = (-1) \begin{bmatrix} 11 \amp -17 \amp -8 \\ -10 \amp 15 \amp 7 \\ 4 \amp -6 \amp -3 \end{bmatrix} \\ \amp = \begin{bmatrix} -11 \amp 17 \amp 8 \\ 10 \amp -15 \amp -7 \\ -4 \amp 6 \amp 3 \end{bmatrix} \end{aligned}
\end{equation*}
The last lemma presented here relates determinants to inverse matrices. At the top of the section, we noted where the notion of the determinant might arise and we will formalize this here.
Lemma 2.8.38.
Let
\(A\) be an
\(n \times n\) matrix. The matrix is invertible if and only if
\(\det(A) \neq 0\text{.}\)
Proof.
First, the forward direction: If
\(\det(A) \neq 0\text{,}\) then from
Lemmaย 2.8.36, then
\(A^{-1}\) can be found.
Next, the revsere direction: If \(\det(A) = 0 \text{,}\) we will use proof by contradiction and assume that \(A^{-1}\) exists that is
\begin{equation*}
A A^{-1} = I
\end{equation*}
If we take the determinant of both sides then
\begin{equation*}
\begin{aligned} \det(A A^{-1}) \amp = \det(I) \\ \det(A) \det(A^{-1}) \amp = 1 \\ \end{aligned}
\end{equation*}
Now, use \(\det(A)=0\) and the above becomes
\begin{equation*}
0 \cdot \det(A^{-1}) = 1
\end{equation*}
which is a contradiction, therefore the assumption that \(A\) has a inverse is false.
Subsection 2.8.10 Cramerโs Rule
This is an optional section that is an interesting use of the determinant. Cramerโs Rule is a method for solving the standard matrix equation
\begin{equation*}
A \boldsymbol{x} = \boldsymbol{b}
\end{equation*}
if \(A\) is an invertible \(n \times n\) matrix.
Let \(A_j\) be the matrix \(A\) with the \(j\)th column of \(A\) is replaced with the vector \(\boldsymbol{b}\text{.}\) Then
\begin{equation*}
\boldsymbol{x} = \frac{\det(A_j)}{\det(A)}
\end{equation*}
Before looking at why this works, letโs redo
Exampleย 1.3.10 using Cramerโs rule.
Example 2.8.39.
Find the solution to \(A\boldsymbol{x} = \boldsymbol{b}\) if
\begin{equation*}
A = \begin{bmatrix} 4 \amp 0 \amp -1 \\ 1 \amp 3 \amp 2 \\ 0 \amp 3 \amp 5 \end{bmatrix} \qquad \qquad \boldsymbol{b} = \begin{bmatrix} 0 \\ 3 \\ 14 \end{bmatrix}
\end{equation*}
using Cramerโs rule.
Solution.
First, we create the matrices
\(A_1, A_2\) and
\(A_3\) which replaces the first, second and third vectors with
\(\boldsymbol{b}\)
\begin{equation*}
\begin{aligned} A_1 \amp = \begin{bmatrix} 0 \amp 0 \amp -1 \\ 3 \amp 3 \amp 2 \\ 14 \amp 3 \amp 5 \end{bmatrix} \\ A_2 \amp = \begin{bmatrix} 4 \amp 0 \amp -1 \\ 1 \amp 3 \amp 2 \\ 0 \amp 14 \amp 5 \end{bmatrix} \\ A_3 \amp = \begin{bmatrix} 4 \amp 0 \amp 0 \\ 1 \amp 3 \amp 3 \\ 0 \amp 3 \amp 14 \end{bmatrix} \end{aligned}
\end{equation*}
Next, we find the deterimants of these matrix as well as \(A\)
\begin{equation*}
\begin{aligned} \det(A) \amp = 33 \amp \det(A_1) \amp = 33 \amp \det(A_2) \amp = -66 \amp \det(A_3) \amp = 132 \end{aligned}
\end{equation*}
Next, we find the solution with
\begin{equation*}
\begin{aligned} x_1 \amp = \frac{\det(A_1)}{\det(A)} = \frac{33}{33} = 1 \\ x_2 \amp = \frac{\det(A_2)}{\det(A)} = \frac{-66}{33} = -2 \\ x_3 \amp = \frac{\det(A_3)}{\det(A)} = \frac{132}{33} = 4 \end{aligned}
\end{equation*}
Cramerโs rule seems like a nice approach to solving linear systems, however, it is computationally intensive. For this (small) 3 by 3 example, 4 determinants need to be computed. If done by hand, there are a lot calculations to do. For larger systems, this can be untenable even if software is used. Row operations are generally the method to use to do in fewer steps.
Also, if
\(A\) is not square then this method cannot be used. However, it is nice to have an alternative method for solving some linear systems.
Justification for why Cramerโs Rule works?????