If
\begin{equation*}
A = \begin{bmatrix}
1 \amp -3 \amp 7 \\
2 \amp 0 \amp 5
\end{bmatrix}
\end{equation*}
then
\begin{equation*}
A^{\intercal} =
\begin{bmatrix}
1 \amp 2 \\ -3 \amp 0 \\ 7 \amp 5
\end{bmatrix}
\end{equation*}
Section 5.1 Matrix Properties and Operations
Although we have spent some time with matrices earlier in the text, we summarize these and they will be important throughout this chapter.
Subsection 5.1.1 The Matrix Transpose
Let \(A\) be an \(m\) by \(n\) matrix. The matrix transpose of \(A\) is denoted \(A^{\intercal}\) and
\begin{equation*}
(A^{\intercal})_{i,j} = A_{j,i}
\end{equation*}
Example 5.1.1.
Theorem 5.1.2. Properties of the matrix transpose.
The following are properties of the transpose. Let \(A\) be an \(m\) by \(n\) matrix (unless more restricted as noted).
-
\((A^{\intercal})^{\intercal} = A\text{.}\) That is, the transpose of the transpose is the original matrix back.
-
\((A+B)^{\intercal} = A^{\intercal} + B^{\intercal}\text{.}\) This is one of the properties of linearity.
-
\((AB)^{\intercal} = B^{\intercal} A^{\intercal}\text{.}\)
-
\((cA)^{\intercal} = c A^{\intercal}\text{.}\) And this is the other linearly property.
If, in addition, the matrix \(A\) is square, then
-
\((A^{-1})^T=(A^T)^{-1}\text{.}\)
-
The eigenvalues of \(A\) are the same as the eigenvalues of \(A^T\text{.}\) Also, the characteristic polynomial of \(A\) and \(A^{\intercal}\) are the same.
Subsection 5.1.2 Symmetric Matrices
We have seen throughout this book examples where a matrix is symmetric in some sense. However, there is a precise definition.
Definition 5.1.3.
A matrix is symmetric if it is square and if \(A= A^{\intercal}\text{.}\)
Example 5.1.4.
The following matrix is symmetric,
\begin{equation*}
A = \begin{bmatrix}
1 \amp 2 \amp 0 \\
2 \amp 3 \amp -4 \\
0 \amp -4 \amp -5
\end{bmatrix}
\end{equation*}
and the following is not
\begin{equation*}
A = \begin{bmatrix}
1 \amp 0 \amp -3 \\
0 \amp 3 \amp 1 \\
-3 \amp 1 \amp 0 \\
0 \amp 0 \amp 0
\end{bmatrix}
\end{equation*}
since it is not square.
Notice that a symmetric matrix has the property that it symmetric over the main diagonal (running from the upper left to lower right).
Subsubsection 5.1.2.1 Eigenvalues of Symmetric Matrices
If we follow the techniques in Section 4.1, we can find the eigenvalues and eigenvectors of
\begin{equation*}
A = \begin{bmatrix}
3 \amp 2 \amp 4 \\
2 \amp 0 \amp 2 \\
4 \amp 2 \amp 3
\end{bmatrix}
\end{equation*}
then the eigenvalues are \(\lambda=8,-1,-1\) and the corresponding eigenvectors are
\begin{align*}
\vec{v}_1 \amp = \begin{bmatrix}
2 \\ 1 \\ 2
\end{bmatrix} \amp \vec{v}_2 \amp =
\begin{bmatrix}
-1 \\ 0 \\ 1
\end{bmatrix} \amp \vec{v}_3 \amp =
\begin{bmatrix}
-1 \\ 2 \\ 0
\end{bmatrix}
\end{align*}
which are both real eigenvalues and eigenvectors. This fact is (as shown in the next lemma) that the matrix is symmetric.
Lemma 5.1.5.
Proof.
\begin{align}
A\vec{v} \amp = \lambda \vec{v}\tag{5.1.1}\\
\amp \quad \quad \text{take the complex conjugate of both sides}\notag\\
\overline{A} \overline{\vec{v}} \amp = \overline{\lambda} \overline{\vec{v}}\notag\\
\amp \quad \quad \text{and since $A$ is real}\notag\\
A \overline{\vec{v}} \amp =
\overline{\lambda} \overline{\vec{v}}\tag{5.1.2}
\end{align}
\begin{align}
\overline{\vec{v}}^{\intercal} A\vec{v} \amp = \overline{\vec{v}}^{\intercal}
(\lambda \vec{v})\notag\\
\amp = \lambda \overline{\vec{v}}^{\intercal} \vec{v}\notag\\
\amp \quad\quad \text{and solving for $\lambda$,}\notag\\
\lambda \amp = \frac{\overline{\vec{v}}^{\intercal}
A \vec{v}}{\overline{\vec{v}}^{\intercal}\vec{v}}\tag{5.1.3}
\end{align}
\begin{align*}
\vec{v}^{\intercal} \overline{A}\overline{\vec{v}} \amp = \vec{v}^{\intercal}
(\overline{\lambda} \overline{\vec{v}}) \\
\amp \quad \quad\text{and if we take the transpose of both sides of this}\\
(\vec{v}^{\intercal} \overline{A}\overline{\vec{v}})^{\intercal} \amp =
(\vec{v}^{\intercal} \overline{\lambda} \overline{\vec{v}})^{\intercal} \notag \\
\overline{\vec{v}}^{\intercal} A^{\intercal} \vec{v} \amp = \overline{\lambda}
\overline{\vec{v}}^{\intercal} \vec{v}
\end{align*}
\begin{equation}
\overline{\lambda} = \frac{\overline{\vec{v}}^{\intercal} A
\vec{v}}{\overline{\vec{v}}^{\intercal} \vec{v} }\tag{5.1.4}
\end{equation}
and since ((5.1.3)) is the same as ((5.1.4)), then \(\lambda = \overline{\lambda}\) so the eigenvalue is real.
Since the eigenvalue of a symmetric matrix is real and the matrix \(A\) is real, then the eigenvectors are also real.
The fact that symmetric matrices have real eigenvalues with be used later in this section.
Subsection 5.1.3 Orthogonal Matrices
An orthogonal matrix, \(Q\) is a square matrix that satisfies \(QQ^{\intercal}
=Q^{\intercal}Q=I\text{.}\)
The properties include:
-
The columns of \(Q\) are orthogonal unit vectors.
-
The rows of \(Q\) are orthogonal unit vectors.
-
\begin{equation*} 1 = \det I = \det (QQ^{\intercal}) = \det(Q) \det(Q^{\intercal}) = \det(Q)^2 \end{equation*}
Example 5.1.6.
The following are examples of 2 by 2 orthogonal matrices:
\begin{align*}
\begin{bmatrix}
0 \amp 1 \\
-1 \amp 0
\end{bmatrix}, \qquad \begin{bmatrix}
\frac{\sqrt{2}}{2} \amp -\frac{\sqrt{2}}{2} \\
\frac{\sqrt{2}}{2} \amp \frac{\sqrt{2}}{2}
\end{bmatrix}
\end{align*}
And the following are 3 by 3 orthogonal matrices:
\begin{align*}
\begin{bmatrix}
\frac{2}{3} \amp -\frac{2}{3} \amp \frac{1}{3}\\[4pt]
-\frac{1}{\sqrt{5}} \amp 0 \amp \frac{2}{\sqrt{5}} \\[4pt]
\frac{4}{3\sqrt{5}} \amp \frac{5}{3\sqrt{5}} \amp \frac{2}{3\sqrt{5}}
\end{bmatrix} \amp\amp
\begin{bmatrix}
0 \amp 1 \amp 0 \\
1 \amp 0 \amp 0 \\
0 \amp 0 \amp 1
\end{bmatrix}
\end{align*}
The matrix on the right is an example of a permutation matrix. Every permutation matrix is an orthogonal matrix.
Subsection 5.1.4 Upper- and Lower-Triangular Matrices
An upper triangle matrix is a matrix (not-necessarily square) such that all elements below the diagonal are 0. A lower triangular matrix is a matrix such that all elements above the diagonal are 0. More precisely,
Definition 5.1.7.
A upper-triangular matrix is an \(m\) by \(n\) matrix \(A\text{,}\) such that \(a_{i,j}=0\) if \(j \lt i\text{.}\) A lower triangular matrix is an \(m\) by \(n\) matrix \(A\) such that \(a_{i,j}=0\) if \(j \gt i\text{.}\)
Example 5.1.8.
\begin{align*}
L \amp = \begin{bmatrix}
3 \amp 0 \amp 0 \\
2 \amp -1 \amp 0 \\
0 \amp 5 \amp 2
\end{bmatrix} \amp
U \amp = \begin{bmatrix}
1 \amp -7 \amp -5 \\
0 \amp 3 \amp 2 \\
0 \amp 0 \amp -3
\end{bmatrix}
\end{align*}
Hopefully that it’s clear that a matrix in echelon form defined in Definition Definition 1.3.1 is also upper triangular.
Subsubsection 5.1.4.1 Back Substitution
An advantage to having an upper- or lower-triangular matrix is that solving the matrix is quite straightforward. In the case of a upper-triangular matrix, \(U\) the steps that solve \(U\vec{x}=\vec{b}\) is called back substitution. We will show the steps in an example.
Example 5.1.9.
If
\begin{align*}
U \amp = \begin{bmatrix}
1 \amp -7 \amp -5 \\
0 \amp 3 \amp 2 \\
0 \amp 0 \amp -3
\end{bmatrix} \amp \vec{b} \amp = \begin{bmatrix}
2 \\ -5 \\ 6
\end{bmatrix},
\end{align*}
solve \(U\vec{x}=\vec{b}\text{.}\)
Solution.
To illustrate back substitution, we write out the matrix equation as the corresponding equations:
\begin{align*}
x_1 -7x_2 -5x_3 \amp = 2 \\
3x_2 +2x_3 \amp = -5 \\
-3 x_3 \amp = -6
\end{align*}
and solving the 3rd equation above, \(x_3=2\text{.}\) This is then substituted into the 2nd equation to get:
\begin{equation*}
3x_2 +2(2) = -5
\end{equation*}
and solving leads to \(x_2 =-3\text{.}\) And then lastly, \(x_2\) and \(x_3\) are substituted into the first equation to get:
\begin{equation*}
x_1 -7(-3)-5(2)=2
\end{equation*}
\begin{equation*}
\vec{x} = \begin{bmatrix}
13 \\ -3 \\ 2
\end{bmatrix}
\end{equation*}
In general, back substitution works the same way.
Remark 5.1.10.
To solve the matrix equation \(U\vec{x}=\vec{b}\text{,}\) where \(U\) is an \(n\) by \(n\text{,}\) non-singular, upper-triangular matrix, first
\begin{equation*}
x_n = \frac{b_n}{a_{nn}}
\end{equation*}
and assume that \(x_{j+1}, x_{j+2}, \ldots, x_n\) are known. Then,
\begin{equation*}
x_j = \frac{1}{a_{jj}} \bigl(b_j - \sum_{k=j+1}^n a_{jk} x_k\bigr) \qquad \text{for
$j=n-1,n-2,\ldots,1$}
\end{equation*}
and it is important that the steps are performed in reverse order. This technique is called back substitution.
Subsubsection 5.1.4.2 Forward Substitution
Similar to back substitution, if a matrix is in lower-diagonal form, then the matrix equation \(L\vec{y}=\vec{c}\) can be solved in a straightforward manner. We first look at an example:
Example 5.1.11.
Solve \(L\vec{y}=\vec{c}\) if
\begin{align*}
L \amp = \begin{bmatrix}
3 \amp 0 \amp 0 \\
2 \amp -1 \amp 0 \\
0 \amp 5 \amp 2
\end{bmatrix} \amp \vec{c} \amp = \begin{bmatrix}
9 \\ 8 \\2 \end{bmatrix}
\end{align*}
Solution.
We first write down the equations that correspond to these:
\begin{align*}
3y_1 \amp = 9 \\
2y_1-y_2 \amp = 8 \\
5y_2 + 2y_3 \amp = 2
\end{align*}
Solving the first equation is \(y_1=3\text{.}\) Then substitute this into the second equation to get:
\begin{equation*}
2(3) - y_2 = 8
\end{equation*}
which has the solution \(y_2=-2\text{.}\) Lastly, we substitute these into the last equation to get:
\begin{equation*}
5(-2) +2y_3 = 2
\end{equation*}
which has the solution \(y_3=6\text{.}\) Thus the solution to the matrix equation is
\begin{equation*}
\vec{y} =\begin{bmatrix}
3 \\ -2 \\ 6
\end{bmatrix}
\end{equation*}
Notice in this example, we solved the unknown’s in the \(\vec{y}\) vector in order. This is because of the structure of the lower-triangular matrix and hence why this technique is called forward substitution.
Remark 5.1.12.
To solve the matrix equation \(L\vec{y}=\vec{c}\text{,}\) where \(L\) is an \(n\) by \(n\text{,}\) nonsingular, lower-triangular matrix, first,
\begin{equation*}
y_1 = \frac{c_1}{a_{11}}
\end{equation*}
and then assume that \(y_1, y_2, \ldots, y_{j-1}\) are known. Then
\begin{equation*}
y_j = \frac{1}{a_{jj}} \bigl(b_j - \sum_{k=1}^{j-1} a_{jk} x_k\bigr) \qquad \text{for
$j=2,3, \ldots n$}
\end{equation*}
and note that in this case, the steps are performed in order. This technique is called forward substitution.
