Definition 5.3.1.
\begin{equation*}
A = P D P^{-1}
\end{equation*}
Section 5.3 Diagonalization of Matrices
Consider a matrix \(A\text{.}\) There are many application where the power of the matrix, \(A^n\) is helpful. One such cases is at the end of this this chapter. One way to approach this is that the \(n\)th power is just the matrix product with itself \(n\) times or
\begin{align*}
A^2 \amp = A A \amp A^3 \amp = A A A
\end{align*}
and this can be extended to any positive integer power. However, finding the 50th power may not be practical.
To make this an easier task, let’s assume that we can write \(A\) in the form:
\begin{equation}
A = P D P^{-1}\tag{5.3.1}
\end{equation}
\begin{align*}
A^3 \amp = A A A \\
\amp = (P D P^{-1}) (P D P^{-1}(P D P^{-1})\\
\amp = P D^3 P^{-1}
\end{align*}
and \(D^3\) is easy to find because it is a diagonal matrix.
If it’s possible to factor \(A\) as in ((5.3.1)), then we call a matrix diagonalizable. Here’s a formal definition.
Before explaining how starting with a matrix \(A\text{,}\) we can find matrices \(D\) and \(P\text{,}\) let’s look at one that is factored in this way.
Example 5.3.2.
Let
\begin{align*}
A \amp=
\begin{bmatrix}
1 \amp 2 \\ 4 \amp 3
\end{bmatrix}
\amp P\amp = \begin{bmatrix}
1 \amp 1 \\
-1 \amp 2
\end{bmatrix} \amp D \amp = \begin{bmatrix}
-1 \amp 0 \\
0 \amp 5
\end{bmatrix}
\end{align*}
Show that \(A = P D P^{-1}\text{.}\)
Solution.
First, recall that
\begin{align*}
P^{-1} \amp = \frac{1}{|P|} \begin{bmatrix}
2 \amp -1 \\
1 \amp 1
\end{bmatrix} = \frac{1}{3} \begin{bmatrix}
2 \amp -1 \\
1 \amp 1
\end{bmatrix}
\end{align*}
and
\begin{equation*}
P D = \begin{bmatrix}
-1 \amp 5 \\
1 \amp 10
\end{bmatrix}
\end{equation*}
then
\begin{equation*}
P D P^{-1} = \begin{bmatrix}
1 \amp 2 \\
4 \amp 3
\end{bmatrix}
\end{equation*}
which shows in this particular example that \(A=PDP^{-1}\text{.}\)
Theorem 5.3.3.
An \(n\) by \(n\) matrix \(A\) is diagonalizable if and only if it has \(n\) linearly independent eigenvectors. The matrix \(P\) is a matrix of the eigenvectors and the matrix \(D\) is the diagonal matrix of eigenvalues.
Proof.
Thus we want to show that
\begin{align*}
A \amp = P D P^{-1} \amp\amp \text{or} \\
A P \amp = P D
\end{align*}
since \(P\) is invertible. Let
\begin{align*}
P \amp = \begin{bmatrix}
\vec{v}_1 \amp \vec{v}_2 \amp \cdots \amp \vec{v}_n
\end{bmatrix}
\amp D \amp = \begin{bmatrix}
\lambda_1 \amp 0 \amp 0 \amp \cdots \amp 0 \\
0 \amp \lambda_2 \amp 0 \amp \cdots \amp 0 \\
0 \amp 0 \amp \lambda_3 \amp \cdots \amp 0 \\
\vdots \amp \vdots \amp \vdots \amp \amp \vdots \\
0 \amp 0 \amp 0 \amp \cdots \amp \lambda_n
\end{bmatrix}
\end{align*}
\begin{align*}
A P \amp = \begin{bmatrix}
A \vec{v}_1 \amp A \vec{v}_2 \amp \cdots A \vec{v}_n
\end{bmatrix} \\
\amp = \begin{bmatrix}
\lambda_1 \vec{v}_1 \amp \lambda_2 \vec{v}_2 \amp \cdots \amp \lambda_n \vec{v}_n
\end{bmatrix} \\
\amp = \begin{bmatrix}
\vec{v}_1 \lambda_1 \amp \vec{v}_2 \lambda_2 \amp \cdots \amp \vec{v}_n \lambda_n
\end{bmatrix}
= P D
\end{align*}
Example 5.3.4.
Is the matrix
\begin{equation*}
A = \begin{bmatrix}
1 \amp -1 \\
2 \amp 4
\end{bmatrix}
\end{equation*}
Solution.
To check this, we need to find the eigenvalues and eigenvectors. First, find the eigenvalues by solving \(|A-\lambda I|=0\) or \(\lambda^2-5\lambda+6=(\lambda-3)(\lambda-2)=0\) or \(\lambda_1=2\) and \(\lambda_2=3\) The associated eigenvectors are
\begin{align*}
\vec{v}_1 \amp= \begin{bmatrix}
1 \\ -1
\end{bmatrix}, \amp \vec{v}_2 \amp = \begin{bmatrix}
1 \\ -2
\end{bmatrix}
\end{align*}
(The steps to find these aren’t shown, but follow the steps in section Section 4.1). Since there are 2 linearly independent eigenvectors, this vector is diagonalizable and
\begin{align*}
P \amp = \begin{bmatrix}
1 \amp 1 \\ -1 \amp -2
\end{bmatrix} \amp D \amp = \begin{bmatrix}
2 \amp 0 \\
0 \amp 3
\end{bmatrix}
\end{align*}
Example 5.3.5.
Is the matrix
\begin{equation*}
A =
\begin{bmatrix}
1 \amp 0 \amp 0 \\
3 \amp -2 \amp 6 \\
2 \amp 1 \amp 3
\end{bmatrix}
\end{equation*}
Solution.
From example Example 4.1.10, we found that \(A\) has eigenvalues \(\lambda_1 = 1, \lambda_2 = 4, \lambda_3 = -3\text{.}\) It also has eigenvectors \(\vec{v}_1 =[-4\;\;2\;\;3]^{\intercal} \text{,}\) \(\vec{v}_2 =[0\;\;1\;\;1]^{\intercal}\) and \(\vec{v}_3=[0\;\;-6\;\;1]^{\intercal}\text{.}\) Therefore the matrices:
\begin{align*}
P \amp = \begin{bmatrix}
-4 \amp 0 \amp 0 \\
2 \amp 1 \amp -6 \\
3 \amp 1 \amp 1
\end{bmatrix} \amp
D \amp =
\begin{bmatrix}
1 \amp 0 \amp 0 \\ 0 \amp 4 \amp 0 \\ 0 \amp 0 \amp -3
\end{bmatrix}
\end{align*}
satisfy \(A = PDP^{-1}\)
Subsection 5.3.1 Powers of Diagonalizable Matrices
One main reason for writing a matrix in diagonalizable form is that powers of the matrix are easy to compute. Note that if \(A = P^{-1} DP\text{,}\) then
\begin{align*}
A^2 \amp = ( PDP^{-1})(PDP^{-1}) \\
\amp = PD^2P^{-1}
\end{align*}
and by induction:
\begin{equation*}
A^k = PD^k P^{-1}
\end{equation*}
Note that raising the diagonal matrix \(D\) to a power is a simple process .
Example 5.3.6.
Use the fact that \(A\) is diagonalizable to find
\begin{equation*}
\begin{bmatrix}
1 \amp -1 \\
2 \amp 4
\end{bmatrix}^5
\end{equation*}
Solution.
\begin{align*}
P \amp = \begin{bmatrix}
1 \amp 1 \\ -1 \amp -2
\end{bmatrix} \amp D \amp = \begin{bmatrix}
2 \amp 0 \\
0 \amp 3
\end{bmatrix}
\end{align*}
Then, \(A^5\) can be written
\begin{align*}
A^5 \amp = P D^5 P^{-1} \\
\amp = \begin{bmatrix}
1 \amp 1 \\
-1 \amp -2
\end{bmatrix} \begin{bmatrix}
2 \amp 0 \\
0 \amp 3
\end{bmatrix}^5 \begin{bmatrix}
2 \amp 1 \\
-1 \amp -1
\end{bmatrix} \\
\amp = \begin{bmatrix}
1 \amp 1 \\
-1 \amp -2
\end{bmatrix} \begin{bmatrix}
32 \amp 0 \\
0 \amp 243
\end{bmatrix} \begin{bmatrix}
2 \amp 1 \\
-1 \amp -1
\end{bmatrix} \\
\amp = \begin{bmatrix}
-179 \amp -211 \\
422 \amp 454
\end{bmatrix}
\end{align*}
Subsection 5.3.2 Similar Matrices
Definition 5.3.7.
Two \(n\) by \(n\) matrices \(A\) and \(B\) are said to be similar if there exists an invertible matrix \(S\) such that
\begin{equation*}
B = S^{-1} A S
\end{equation*}
Theorem 5.3.8.
If \(A\) and \(B\) are similar \(n\) by \(n\) matrices, then the following are identical for the two matrices:
Theorem 5.3.9.
If \(A\) and \(B\) are similar \(n\) by \(n\) matrices, then \(A\) and \(B\) have the same characteristic equation and therefore have the same eigenvalues.
Proof.
Let \(A\) satisfy the characteristic equation
\begin{equation*}
\det(A - \lambda I) = 0
\end{equation*}
\begin{align*}
\det(S^{-1} B S - \lambda I) \amp = 0 \qquad \text{Since $S^{-1}S=I=S^{-1} IS$}\\
\det(S^{-1} B S - \lambda S^{-1} I S) \amp = 0 \qquad\text{Factoring out a $S^{-1}$ from the left and $S$ from the right}\\
\det (S^{-1}(B - \lambda S I S^{-1}) S) \amp = 0 \\
\det(S^{-1}) \det(B-\lambda I) \det(S) \amp = 0\\
\amp \qquad \text{since $\det(S^{-1}) \det(S)= \det(S^{-1}S)=\det(I)= 1$}\\
\det(B - \lambda I) \amp = 0
\end{align*}
so the characteristic equation is identical, thus the eigenvalues are the same.
Example 5.3.10.
Show that
\begin{align*}
A \amp = \begin{bmatrix}
1 \amp -1 \\ 2 \amp 4
\end{bmatrix} \amp B \amp = \begin{bmatrix}
2 \amp 2 \\ 0 \amp 3
\end{bmatrix}
\end{align*}
are similar matrices with
\begin{equation*}
S = \begin{bmatrix}
-1 \amp 1 \\ 1 \amp 0
\end{bmatrix}
\end{equation*}
Solution.
First, using the formula for the inverse of a \(2 \times 2\) matrix,
\begin{equation*}
S^{-1} = \begin{bmatrix}
0 \amp 1 \\ 1 \amp 1
\end{bmatrix}
\end{equation*}
and
\begin{align*}
S^{-1} A S \amp = \begin{bmatrix}
0 \amp 1 \\ 1 \amp 1
\end{bmatrix} \begin{bmatrix}
1 \amp -1 \\ 2 \amp 4
\end{bmatrix}\begin{bmatrix}
-1 \amp 1 \\ 1 \amp 0
\end{bmatrix} = \begin{bmatrix}
2\amp 2 \\ 0 \amp 3
\end{bmatrix} = B
\end{align*}
Subsection 5.3.3 Similar and Diagonalizable Matrices
It appears that there is a connection to similar and diagonalizable matrices through at least their near identical formulas. Notice that \(A\) and \(B\) are similar if there exists an invertible matrix \(P\) such that
\begin{equation*}
A = S^{-1} B S
\end{equation*}
\begin{equation}
A = P_1D_1 P^{-1}_1\tag{5.3.2}
\end{equation}
for a diagonal matrix \(D_1\text{.}\) Similarly \(B\) is diagonalizable if there exists an invertible \(P_2\) such that
\begin{equation}
B = P_2 D_2 P_{2}^{-1}\tag{5.3.3}
\end{equation}
for diagonal matrix \(D_2\text{.}\) We know that the matrices \(P_1\) and \(P_2\) exist if there are \(n\) linearly independent eigenvectors, but how do you find \(S\text{?}\) Solving for \(D_1\) in (5.3.2) and \(D_2\) in (5.3.3),
\begin{align*}
D_1 \amp = P_1^{-1}A P_1 \amp D_2 \amp = P_2^{-1} B P_2
\end{align*}
Also, since the eigenvalues of \(A\) and \(B\) are the same, we can rearrange the eigenvectors of \(A\) and \(B\) to the same order thus without loss of generality, \(D_1=D_2\) and
\begin{align*}
P_1^{-1} A P_1 \amp = P_2^{-1} B P_2\\
\text{left multiplying by $P_1$ and right multiplying by $P_1^{-1}$}\\
A \amp = P_1 P_2^{-1} B P_2 P_1^{-1}
\end{align*}
thus let
\begin{equation}
S = P_2 P_1^{-1}.\tag{5.3.4}
\end{equation}
Example 5.3.11.
Above we showed that the matrices
\begin{align*}
A \amp = \begin{bmatrix}
1 \amp -1 \\ 2 \amp 4
\end{bmatrix} \amp B \amp = \begin{bmatrix}
2 \amp 2 \\ 0 \amp 3
\end{bmatrix}
\end{align*}
Solution.
Briefly, we need to diagonalize both \(A\) and \(B\text{.}\) In example Example 5.3.4, we found that the eigenvalues of \(A\) are \(\lambda_1=2\) and \(\lambda_2=3\) with associated eigenvectors \(\vec{v}_1=[1\;\;-1]^{\intercal}\) and \(\vec{v}_2=[1\;\;-2]^{\intercal}\text{.}\) Using the techniques of section Section 4.1, the eigenvalues of \(B\) are \(\lambda_1=2\) and \(\lambda_2=3\) with associated eigenvectors \(\vec{v}_1=[1\;\;4]^{\intercal}\) and \(\vec{v}_2=[0\;\;1]^{\intercal}\text{.}\) Letting \(D_1\) and \(P_1\) be the matrices associated with \(A\) and \(D_2\) and \(P_2\text{,}\) those associated with \(B\text{,}\) let
\begin{equation*}
D_1 = D_2 = \begin{bmatrix}
3 \amp 0 \\ 0 \amp 2
\end{bmatrix}
\end{equation*}
and
\begin{align*}
P_1 \amp = \begin{bmatrix}
1 \amp 1 \\
-1 \amp -2
\end{bmatrix}, \amp P_2 \amp = \begin{bmatrix}
2 \amp 1 \\
1 \amp 0
\end{bmatrix}
\end{align*}
\begin{equation*}
S = P_{2} P_1^{-1} = \begin{bmatrix}
2 \amp 1 \\ 1 \amp 0
\end{bmatrix} \begin{bmatrix}
-1 \amp -1\\
2 \amp 1\\
\end{bmatrix}= \begin{bmatrix}
0 \amp -1 \\
-1 \amp -1
\end{bmatrix}
\end{equation*}
And although this \(S\) is not the same \(S\) as Example 5.3.10, this matrix \(S\) is the negative of the inverse of that matrix in Example 5.3.10.
