Skip to main content

Applied Mathematics

Section 1.3 Echelon Forms and Gaussian Elimination

In this section, we formalize the solving of linear systems using matrices. You will learn:
  • Important forms of matrices called row-echelon and reduced row-echelon forms.
  • An algorithm to get any matrix into these forms.
  • How to write the solution to a linear system in these forms.
Recall from SectionΒ 1.2 how to convert any linear system into an augmented coefficient matrix. This will be needed to solve linear systems.

Subsection 1.3.1 Row-Echelon and Reduced Row-Echelon Form

Notice that the strategy laid out above gets the linear system in a nice form for solving for all variables using back substitution. We will now explicitly state what this nice form is.

Definition 1.3.1.

In each row of a matrix, the left-most entry that is non-zero is called the leading term. A matrix is in row-echelon form if
  • there are any rows that are all zero, they are at the bottom of the matrix.
  • each leading term is to the right of the leading term of the row above it (except the first row).

Example 1.3.2. Matrices in Row-Echelon Form.

The following matrices are in row-echelon form:
\begin{align*} \begin{bmatrix} 3 \amp 0 \amp -2 \amp 11 \\ 0 \amp 0 \amp 9 \amp 2 \\ 0 \amp 0 \amp 0 \amp -4 \end{bmatrix} \amp \amp \begin{bmatrix} 0 \amp 0 \amp 2 \amp -5 \amp 9 \amp \pi \\ 0 \amp 0 \amp 0 \amp 9 \amp 2 \amp 0 \\ 0 \amp 0 \amp 0 \amp 0 \amp 0 \amp 7 \end{bmatrix} \amp \amp \begin{bmatrix} 1 \amp 2 \amp 3 \\ 0 \amp 4 \amp 5 \\ 0 \amp 0 \amp 6 \\ 0 \amp 0 \amp 0 \\ 0 \amp 0 \amp 0 \end{bmatrix} \end{align*}

Definition 1.3.3.

A matrix is in reduced row-echelon form if it is echelon form and has the following properties:
In each row of a matrix, the left-most term that is non-zero is called the leading term. A matrix is in echelon form if each leading term is to the right of the leading term of the row above it (except the first row).

Example 1.3.4.

The following matrix are reduced row-echelon form:
\begin{align*} \begin{bmatrix} 1 \amp 0 \amp 0 \amp 9 \\ 0 \amp 0 \amp 1 \amp 3/2 \\ 0 \amp 0 \amp 0 \amp 1 \end{bmatrix} \amp \amp \begin{bmatrix} 0 \amp 0 \amp 1 \amp 0 \amp 3 \amp 0 \\ 0 \amp 0 \amp 0 \amp 1 \amp 9 \amp 0 \\ 0 \amp 0 \amp 0 \amp 0 \amp 0 \amp 1 \end{bmatrix} \amp \amp \begin{bmatrix} 1 \amp 0 \amp 0 \\ 0 \amp 1 \amp 0 \\ 0 \amp 0 \amp 1 \\ 0 \amp 0 \amp 0 \\ 0 \amp 0 \amp 0 \end{bmatrix} \end{align*}
and the following are not in reduced-row echelon form
\begin{align*} \begin{bmatrix} 3 \amp 0 \amp -2 \amp 11 \\ 0 \amp 0 \amp 9 \amp 2 \\ 0 \amp 0 \amp 0 \amp -4 \end{bmatrix} \amp \amp \begin{bmatrix} 0 \amp 0 \amp 1 \amp -5 \\ 0 \amp 1 \amp 0 \amp 9 \\ 0 \amp 0 \amp 0 \amp 0 \end{bmatrix} \amp \amp \begin{bmatrix} 0 \amp 0 \amp 0 \\ 1 \amp 2 \amp 0 \\ 0 \amp 0 \amp 1 \end{bmatrix} \amp \amp \begin{bmatrix} 1 \amp -2 \amp 0 \\ 0 \amp 1 \amp 0 \\ 0 \amp 0 \amp 1 \end{bmatrix} \end{align*}
The first matrix is in echelon form, but the leading coefficients are not 1. The second matrix isn’t in echelon form. The leading coefficient in the 2nd row is to the left of the row above. The third matrix has a row of zeros that is not at the bottom. The fourth matrix doesn’t have the leading term in a column that is zero otherwise.

Subsection 1.3.2 Solving Linear Systems with Matrices

Returing to the solution of linear systems, in the context of matrices, we seek to put an augmented matrix into either row-echelon or reduced row-echelon form. The following strategy for a system with 3 equations and 3 variables is often helpful, however, as we will see not guaranteed.
As we will see, it is desirable to get zeros in parts of the matrix. This is generate done with the following:

Definition 1.3.5.

If there is a 1 in row \(i\text{,}\) the row operation
\begin{equation} -c R_i + R_j \to R_j\tag{1.3.1} \end{equation}
where \(c\) is the value in the \(j\)th row in the same column will put a 0 in that location. This is called the Elimination Opeation.

Remark 1.3.6. Strategy for putting a matrix in Row-Echelon Form.

Consider a general \(3 \times 4\) matrix representating a linear system with 3 equations and 3 variables. The \(\star\) means the value of entry can be any number.
\begin{equation*} \left[\begin{array}{rrr|r} \star \amp \star \amp \star \amp \star \\ \star \amp \star \amp \star \amp \star \\ \star \amp \star \amp \star \amp \star \end{array}\right] \end{equation*}
The first step is to get a 1 in the first column, first row or
\begin{equation*} \left[\begin{array}{rrr|r} 1 \amp \star \amp \star \amp \star \\ \star \amp \star \amp \star \amp \star \\ \star \amp \star \amp \star \amp \star \end{array}\right] \end{equation*}
We next desire to zero out the rest of the first column, which we do with the elimination operation from DefinitionΒ 1.3.5.
Next, we will use this to zero out the 2nd and 3rd rows in the first column to get:
\begin{equation*} \left[\begin{array}{rrr|r} 1 \amp \star \amp \star \amp \star \\ 0 \amp \star \amp \star \amp \star \\ 0 \amp \star \amp \star \amp \star \end{array}\right] \end{equation*}
Next, we seek a 1 in the 2nd row, 2nd column. or
\begin{equation*} \left[\begin{array}{rrr|r} 1 \amp \star \amp \star \amp \star \\ 0 \amp 1 \amp \star \amp \star \\ 0 \amp \star \amp \star \amp \star \end{array}\right] \end{equation*}
And then if we are looking for a matrix in echelon form, zero out the 3rd row, 2nd column. We use the same general row operation in (1.3.1) however using the 2nd row instead or
\begin{equation*} \left[\begin{array}{rrr|r} 1 \amp \star \amp \star \amp \star \\ 0 \amp 1 \amp \star \amp \star \\ 0 \amp 0 \amp \star \amp \star \end{array}\right] \end{equation*}
This matrix is now in row-echelon form. If instead reduced-row-echelon form is desired, in the previous step, the 2nd column, first row can be eliminated and then another set of steps can be used to get the 3rd row in the correct form.

Example 1.3.7. Solving a Linear System.

Use the strategy in RemarkΒ 1.3.6 to put the augmented matrix of the linear system:
\begin{align*} 4x_1 \phantom{+3x_2}- x_3 \amp = 0, \\ x_1+3x_2 +2x_3 \amp = 3, \\ 3x_2 + 5x_3 \amp = 14. \end{align*}
into row-echelon form.
Solution.
First, put the linear system into an augmented coefficient matrix:
\begin{equation*} \left[\begin{array}{rrr|r} 4 \amp 0 \amp -1 \amp 0 \\ 1 \amp 3 \amp 2 \amp 3 \\ 0 \amp 3 \amp 5 \amp 14 \end{array}\right] \end{equation*}
We first try to get a 1 in the first row, first column. The easiest way to do this is to swap the first two rows:
\begin{equation*} R_1 \leftrightarrow R_2 \qquad \left[\begin{array}{rrr|r} 1 \amp 3 \amp 2 \amp 3 \\ 4 \amp 0 \amp -1 \amp 0 \\ 0 \amp 3 \amp 5 \amp 14 \end{array}\right] \end{equation*}
Now eliminate the term in the 2nd row, first column:
\begin{equation*} -4R_1 +R_2 \to R_2 \qquad \left[\begin{array}{rrr|r} 1 \amp 3 \amp 2 \amp 3 \\ 0 \amp -12 \amp -9 \amp -12 \\ 0 \amp 3 \amp 5 \amp 14 \end{array}\right] \end{equation*}
Then try to get a 1 in the second row, second column. You should only use the bottom two rows to do this because using the top row would mess up the structure of the first column. We’ll just divide the second row by \(-12\text{.}\)
\begin{equation*} -\frac{1}{12} R_2 \to R_2 \qquad \left[\begin{array}{rrr|r} 1 \amp 3 \amp 2 \amp 3 \\ 0 \amp 1 \amp \frac{3}{4} \amp 1 \\ 0 \amp 3 \amp 5 \amp 14 \end{array}\right] \end{equation*}
We have introduced fractions, which is generally not desired. An alternative would have been to wait until later to get a 1 in the second row, second column, but often fractions is unavoidable. Next, eliminate the 3rd row, 2nd column.
\begin{equation*} -3 R_2 + R_3 \to R_3 \qquad \left[\begin{array}{rrr|r} 1 \amp 3 \amp 2 \amp 3 \\ 0 \amp 1 \amp \frac{3}{4} \amp 1 \\ 0 \amp 0 \amp \frac{11}{4} \amp 11 \end{array}\right] \end{equation*}
The last step to get this in row-echelon form is to get the 3rd row, 3rd column to be a 1. Multiply the third row by \(4/11\text{.}\)
\begin{equation*} \frac{4}{11} R_3 \to R_3 \qquad \left[\begin{array}{rrr|r} 1 \amp 3 \amp 2 \amp 3 \\ 0 \amp 1 \amp \frac{3}{4} \amp 1 \\ 0 \amp 0 \amp 1 \amp 4 \end{array}\right] \end{equation*}
One of the advantages of putting an augmented coefficient matrix in row-echelon form is that the solution can be found using back substitution, where generally the last variable is solved for in the last equation and then substituted into previous equations. We see this for the previous example.
We now write the matrix ExampleΒ 1.3.7 as a linear system and use back substitution to solve it.
The augmented matrix can be written as the equations:
\begin{align*} x + 3y + 2z \amp = 3 \\ y + \frac{3}{4} z \amp =1 \\ z \amp =4 \end{align*}
The solution for \(z\) is now plugged into the second equation:
\begin{equation*} y + \frac{3}{4} \cdot 4 = 1 \end{equation*}
leading to \(y = 1-3 = -2\text{.}\) Plugging the solutions for \(y\) and \(z\) into the first equation leads to
\begin{equation*} x + 3 (-2) + 2(4) = 3 \end{equation*}
leading to \(x = 1\text{.}\) The solution to this system is the triple \((3,-2,4)\text{.}\)

Example 1.3.8. Solving the Biathlon Problem.

Solve the linear system for the biathlon race in example ExampleΒ 1.1.1
\begin{align*} x_1+x_2\amp =29, \\ \frac{x_1}{6} + \frac{x_2}{18} \amp =\frac{13}{6}, \end{align*}
Solution.
We start with the augmented coefficient matrix:
\begin{equation*} \left[\begin{array}{rr|r} 1 \amp 1 \amp 29 \\ \frac{1}{6} \amp \frac{1}{18} \amp \frac{13}{6} \end{array}\right] \end{equation*}
First to make life easier, let’s multiple the 2nd row by 18 to rid the fractions.
\begin{equation*} 18 R_2 \rightarrow R_2\quad \left[\begin{array}{rr|r} 1 \amp 1 \amp 29 \\ 3 \amp 1 \amp 39 \end{array}\right] \end{equation*}
Then we eliminate the \(x\) in the 2nd equation by doing the following step,
\begin{equation*} -3R_1 + R_2 \rightarrow R_2\qquad\left[\begin{array}{rr|r} 1 \amp 1 \amp 29 \\ 0 \amp -2 \amp -48 \end{array}\right] \end{equation*}
and at this point, the matrix is in row echelon form and we rewrite the matrix in equaiton form:
\begin{align*} x_1 + x_2 \amp = 29 \\ -2x_2 \amp = -48 \end{align*}
And we can solve for \(x_2\) in the last equation using back substitution. \(x_2=24\) and then
\begin{equation*} x_1 + 24 = 29 \end{equation*}
therefore,
\begin{equation*} x_1 = 5 \end{equation*}
So the running part of the race is 5 miles and the biking is 24 miles.

Subsection 1.3.3 Linear Systems and Reduced Row-Echelon Form

Throughout this section, we have used Gauss’s method to take a linear system as an augmented matrix and put the matrix in row echelon form, a convenient matrix for using back substitution to find the solution to the linear system. For the remainder of the section, we will still use row operations, but put the matrix in reduced row-echelon form. Despite the fact that it takes more steps to put in the form, the result is generally easier to find the solution.

Definition 1.3.9.

Solving a linear system by reducing its corresponding augmented matrix to a reduced row-echelon form matrix, then interpreting its solution is called Gauss-Jordon reduction.

Example 1.3.10.

Solve
\begin{align*} 4x_1 - x_3 \amp = 0, \\ x_1+3x_2 +2x_3 \amp = 3, \\ 3x_2 + 5x_3 \amp = 14. \end{align*}
using Gauss-Jordon reduction
Solution.
First put the linear system into an augmented matrix and then use elementary row operations.
\begin{align*} \amp\left[\begin{array}{rrr|r} 4 \amp 0 \amp -1 \amp 0 \\ 1 \amp 3 \amp 2 \amp 3 \\ 0 \amp 3 \amp 5 \amp 14 \end{array} \right]\\ R_1 \leftrightarrow R_2, \qquad\amp \left[\begin{array}{rrr|r} 1 \amp 3 \amp 2 \amp 3 \\ 4 \amp 0 \amp -1 \amp 0 \\ 0 \amp 3 \amp 5 \amp 14 \end{array} \right] \\ -4R_1 + 4R_2 \rightarrow R_2, \qquad \amp \left[\begin{array}{rrr|r} 1 \amp 3 \amp 2 \amp 3 \\ 0 \amp -12 \amp -9 \amp -12 \\ 0 \amp 3 \amp 5 \amp 14 \end{array} \right]\\ -\frac{1}{12} R_2 \to R_2, \qquad \amp \left[\begin{array}{rrr|r} 1 \amp 3 \amp 2 \amp 3 \\ 0 \amp 1 \amp 3/4 \amp 1 \\ 0 \amp 3 \amp 5 \amp 14 \end{array} \right] \end{align*}
At this point when doing Gaussian Elimination, we generally just zeroed out the 3 in the 3rd row, 2nd column. However, to get things in reduced row echelon form, we will zero out the 3 in the first row, 2nd column as well.
\begin{align*} -\begin{array}{r} -3 R_2 + R_1 \rightarrow R_1 \\ -3R_2 + R_3 \rightarrow R_3, \end{array}, \qquad \amp \left[\begin{array}{rrr|r} 1 \amp 0 \amp -1/4 \amp 0 \\ 0 \amp 1 \amp 3/4 \amp 1 \\ 0 \amp 0 \amp 11/4 \amp 11 \end{array} \right] \end{align*}
\begin{align*} \frac{4}{11} R_3 \to R_3, \qquad \amp\amp \left[\begin{array}{rrr|r} 1 \amp 0 \amp -1/4 \amp 0 \\ 0 \amp 1 \amp 3/4 \amp 1 \\ 0 \amp 0 \amp 1 \amp 4 \end{array}\right] \end{align*}
Now we can zero out the first and second rows of the third column using the standard elimination steps.
\begin{align*} \begin{array}{r}-\frac{3}{4} R_3 + R_2\to R_2,\\ \frac{1}{4} R_3 + R_1 \to R_1 \end{array} \qquad \amp\amp \left[\begin{array}{rrr|r} 1 \amp 0 \amp 0\amp 1 \\ 0 \amp 1 \amp 0 \amp -2 \\ 0 \amp 0 \amp 1 \amp 4 \end{array}\right] \end{align*}
And now, like before, we write down the equations from the augmented coefficient matrix:
\begin{align*} x \amp = 1 \\ y \amp = -2 \\ z \amp = 4 \end{align*}
or the triple \((1,-2,4)\text{.}\) And notice that the solution in this case is simply the right hand side of the matrix above.

Example 1.3.11. Solving the Traffic Problem.

Solve the linear system in ExampleΒ 1.1.2 or written in standard form:
\begin{align*} -x_1 + x_4 + x_5 \amp = 250\\ x_3 + x_4 \amp = 350 \\ x_2 + x_5 \amp = 700 \\ x_3 + x_2 \amp = 600 \end{align*}
by writing the augmented coefficient matrix and putting the matrix in reduced row-echelon form.
Solution.
We first write the augmented coefficient matrix:
\begin{align*} \amp \left[\begin{array}{rrrrr|r} -1 \amp 0 \amp 0 \amp 1 \amp 1 \amp 250 \\ 0 \amp 0 \amp 1 \amp 1 \amp 0 \amp 350 \\ 0 \amp 1 \amp 0 \amp 0 \amp 1 \amp 700 \\ 0 \amp 1 \amp 1 \amp 0 \amp 0 \amp 600 \end{array}\right]\\ -R_1 \to R_1 \qquad \amp \left[\begin{array}{rrrrr|r} 1 \amp 0 \amp 0 \amp -1 \amp -1 \amp -250 \\ 0 \amp 0 \amp 1 \amp 1 \amp 0 \amp 350 \\ 0 \amp 1 \amp 0 \amp 0 \amp 1 \amp 700 \\ 0 \amp 1 \amp 1 \amp 0 \amp 0 \amp 600 \end{array}\right]\\ R_2 \leftrightarrow R_3\qquad \amp \left[\begin{array}{rrrrr|r} 1 \amp 0 \amp 0 \amp -1 \amp -1 \amp -250 \\ 0 \amp 1 \amp 0 \amp 0 \amp 1 \amp 700 \\ 0 \amp 0 \amp 1 \amp 1 \amp 0 \amp 350 \\ 0 \amp 1 \amp 1 \amp 0 \amp 0 \amp 600 \end{array}\right]\\ -R_2 +R_4 \to R_4\qquad \amp \left[\begin{array}{rrrrr|r} 1 \amp 0 \amp 0 \amp -1 \amp -1 \amp -250 \\ 0 \amp 1 \amp 0 \amp 0 \amp 1 \amp 700 \\ 0 \amp 0 \amp 1 \amp 1 \amp 0 \amp 350 \\ 0 \amp 0 \amp 1 \amp 0 \amp -1 \amp -100 \end{array}\right]\\ -R_3 +R_4 \to R_4\qquad \amp \left[\begin{array}{rrrrr|r} 1 \amp 0 \amp 0 \amp -1 \amp -1 \amp -250 \\ 0 \amp 1 \amp 0 \amp 0 \amp 1 \amp 700 \\ 0 \amp 0 \amp 1 \amp 1 \amp 0 \amp 350 \\ 0 \amp 0 \amp 0 \amp -1 \amp -1 \amp -450 \end{array}\right]\\ -R_4 \to R_4\qquad \amp \left[\begin{array}{rrrrr|r} 1 \amp 0 \amp 0 \amp -1 \amp -1 \amp -250 \\ 0 \amp 1 \amp 0 \amp 0 \amp 1 \amp 700 \\ 0 \amp 0 \amp 1 \amp 1 \amp 0 \amp 350 \\ 0 \amp 0 \amp 0 \amp 1 \amp 1 \amp 450 \end{array}\right]\\ \begin{array}{r} -R_4 +R_3 \to R_3 \\ R_4 + R_1 \to R_1 \end{array} \qquad \amp \left[\begin{array}{rrrrr|r} 1 \amp 0 \amp 0 \amp 0 \amp 0 \amp 200 \\ 0 \amp 1 \amp 0 \amp 0 \amp 1 \amp 700 \\ 0 \amp 0 \amp 1 \amp 0 \amp -1 \amp -100 \\ 0 \amp 0 \amp 0 \amp 1 \amp 1 \amp 450 \end{array}\right] \end{align*}
The last step is to rewrite the matrix as a linear system. This matrix can be written as:
\begin{align*} x_1 \amp =200 \\ x_2 + x_5 \amp =700\\ x_3 -x_5 \amp =-100 \\ x_4 + x_5 \amp =450 \end{align*}
And notice that we can solve for \(x_1, x_2, x_3, x_4\text{,}\) but not \(x_5\text{,}\) which is a free variable or
\begin{align*} x_1 \amp =200 \\ x_2 \amp = -x_5 + 700\\ x_3 \amp= x_5 -100 \\ x_4 \amp= -x_5 + 450 \end{align*}
It may seem odd that there are multiple solutions to this problem, at first glance, one should know how many travels are travelling around the city. However, notice that the original problem has 5 variables and only 4 equations. Although there are unique solutions to such systems, generally if there are more variables than equations, there are either no solutions or multiple solutions.
Additionally, on a more global scale for the map shown above there are \(350+500+200=1050\) cars entering the area and \(250+400+200+x_1=850+x_1\) leaving and we don’t know how many cars are leaving the area. If we also include another equation that would balance these (and that makes sense), then there would now be 5 equations and probably would give a unique solution.

Subsection 1.3.4 Describing the Solution Set

So far we have seen linear system with solution sets with different number of elements. The no solution means that the solution set is empty, the unique solution means that there is one element in the set. This was the case in ExampleΒ 1.3.8 and in ExampleΒ 1.3.11, there is a free variable resulting in infinitely many solutions. We examine this last case in more detail.
For example, if we reduce the linear system:
\begin{align*} \amp \amp x\phantom{+2y} + 2z \amp = -6 \\ \amp \amp 2x-y\phantom{+3z}\amp = 1 \\ \amp \amp y+4z\amp = -13 \end{align*}
First, write the linear system as a augmented matrix as
\begin{equation*} \left[\begin{array}{rrr|r} 1 \amp 0 \amp 2 \amp -6 \\ 2 \amp -1 \amp 0 \amp 1 \\ 0 \amp 1 \amp 4 \amp -13 \end{array}\right] \end{equation*}
Eliminate the 2 in the first column, second row.
\begin{equation*} -2R_1+R_2 \rightarrow R_2 \qquad \left[\begin{array}{rrr|r} 1 \amp 0 \amp 2 \amp -6 \\ 0 \amp -1 \amp -4 \amp 13 \\ 0 \amp 1 \amp 4 \amp -13 \end{array}\right] \end{equation*}
Eliminate the 1 in the 3rd row, 2nd column.
\begin{align*} R_2+R_3 \rightarrow R_3 \qquad \amp \left[\begin{array}{rrr|r} 1 \amp 0 \amp 2 \amp -6 \\ 0 \amp -1 \amp -4 \amp 13 \\ 0 \amp 0 \amp 0 \amp 0 \end{array}\right] \\ -R_2 \to R_2 \qquad \amp \left[\begin{array}{rrr|r} 1 \amp 0 \amp 2 \amp -6 \\ 0 \amp 1 \amp 4 \amp -13 \\ 0 \amp 0 \amp 0 \amp 0 \end{array}\right] \end{align*}
This is now in reduced row echelon form and we rewrite as a linear system as
\begin{align*} x\phantom{+2y} + 2z \amp = -6\\ y +4z \amp = -13 \end{align*}
where the last row of the matrix isn’t written, because \(0=0\) conveys no relevant information. Notice in this case, we can solve the first equation for \(x\) and the second equation for \(y\) in terms of \(z\) as
\begin{align*} x \amp = -2z-6 \\ y \amp = -4z -13 \end{align*}
In this case \(z\) is a free variable and we can write the set of points that solve this in the following way:
\begin{equation*} \{ (-2z-6,-4z-13,z)\; |\; z \in \mathbb{R}\} \end{equation*}
and now we have a lot of information about the solution set. For example, if \(z=0\text{,}\) then \(x=-6, y=-13\text{,}\) so the point \((-6,-13,0)\) is in the solution set. Similarly if \(z=-3\text{,}\) then \((0,-1,-3)\) is also a point in the solution set and any point given a value of \(z\) .
We also have a sense of the number of solutions and since \(z\) can take on any real number, there are an infinite number of triples \(3\)-tuples in this solution set.
Also, the form of the solution set above is in set builder form. That is is shows how to create the set of points in the solution.

Definition 1.3.12.

Any non-leading variable in a linear system in echelon form is called a free variable or parameter.
The next example shows that a linear system can have more than one free variable.

Example 1.3.13.

Consider the linear system
\begin{align*} x_1\phantom{+2x_3} + 3x_3 -9 x_4 + 11 x_5 \amp = 14, \\ 2x_3 \phantom{-9x_4} +\phantom{1} 4x_5 \amp = 10, \\ 3x_5 \amp = 27, \end{align*}
Write the solution set in terms of the free variables.
Solution.
First, let’s put this in augmented matrix form as
\begin{equation*} \left[\begin{array}{rrrrr|r} 1 \amp 0 \amp 3 \amp -9 \amp 11 \amp 14 \\ 0 \amp 0 \amp 2 \amp 0 \amp 4 \amp 10 \\ 0 \amp 0 \amp 0 \amp 0 \amp 3 \amp 27 \end{array}\right] \end{equation*}
Notice that this is already in row-echelon form, but we will do a couple of row operation steps to get it into reduced row echelon form.
\begin{align*} \begin{array}{r} \frac{1}{3} R_3 \to R_3\\ \frac{1}{2} R_2 \to R_2 \end{array} \qquad \amp \left[\begin{array}{rrrrr|r} 1 \amp 0 \amp 3 \amp -9 \amp 11 \amp 14 \\ 0 \amp 0 \amp 1 \amp 0 \amp 2 \amp 5 \\ 0 \amp 0 \amp 0 \amp 0 \amp 1 \amp 9 \end{array}\right] \\ \begin{array}{r} -3R_2 + R_1 \to R_1 \end{array} \qquad \amp \left[\begin{array}{rrrrr|r} 1 \amp 0 \amp 0 \amp -9 \amp 5 \amp -1 \\ 0 \amp 0 \amp 1 \amp 0 \amp 2 \amp 5 \\ 0 \amp 0 \amp 0 \amp 0 \amp 1 \amp 9 \end{array}\right] \\ \begin{array}{r} -2R_3 + R_2 \to R_2 \\ -5R_3 + R_1 \to R_1 \end{array} \qquad \amp \left[\begin{array}{rrrrr|r} 1 \amp 0 \amp 0 \amp -9 \amp 0 \amp -46 \\ 0 \amp 0 \amp 1 \amp 0 \amp 0 \amp -13 \\ 0 \amp 0 \amp 0 \amp 0 \amp 1 \amp 9 \end{array}\right] \end{align*}
which is now in reduced row-echelon form. Note that from here we can see that the leading variables are \(x_1, x_3\) and \(x_5\) and the remaining variables \(x_2\) and \(x_4\) are free. Converting this matrix to equations results in
\begin{align*} x_1 -9x_4 \amp = -46 \\ x_3 \amp = -13 \\ x_5 \amp = 9 \end{align*}
and solving the first equation is
\begin{align*} x_1\amp = 9x_4 -46 \\ x_3 \amp = -13 \\ x_5 \amp = 9 \end{align*}
We can now write down the solution in the same form (set-builder) as before as
\begin{equation*} \{ (x_1,x_2,x_3,x_4,x_5) = (9x_4-46,x_2,-13,x_4,9) \; | \; x_2, x_4 \in \mathbb{R}\} \end{equation*}
We now find the solution to the Hydrazine chemistry problem in Example ExampleΒ 1.1.4

Example 1.3.14.

Find the solution set of the linear system in ExampleΒ 1.1.4
\begin{align*} k_1\phantom{+2k_2}-2k_3\phantom{+2k_4} \amp = 0, \\ 3k_1+2k_2-4k_3-2k_4 \amp = 0, \\ 2k_2\phantom{+2x_3}-\phantom{2}k_4 \amp = 0, \end{align*}
Solution.
We will first put this into an augmented matrix, then use Gauss’s method,
\begin{equation*} \left[\begin{array}{rrrr|r} 1 \amp 0 \amp -2 \amp 0 \amp 0 \\ 3 \amp 2 \amp -4 \amp -2 \amp 0 \\ 0 \amp 2 \amp 0 \amp -1 \amp 0 \end{array}\right] \end{equation*}
\begin{equation*} -3 R_1 + R_2 \to R_2 \qquad \left[\begin{array}{rrrr|r} 1 \amp 0 \amp -2 \amp 0 \amp 0 \\ 0 \amp 2 \amp 2 \amp -2 \amp 0 \\ 0 \amp 2 \amp 0 \amp -1 \amp 0 \end{array}\right] \end{equation*}
\begin{equation*} -R_2 + R_3 \to R_3 \qquad \left[\begin{array}{rrrr|r} 1 \amp 0 \amp -2 \amp 0 \amp 0 \\ 0 \amp 2 \amp 2 \amp -2 \amp 0 \\ 0 \amp 0 \amp -2 \amp 1 \amp 0 \end{array}\right] \end{equation*}
\begin{equation*} \begin{array}{r} \frac{1}{2} R_2 \to R_2 \\ -\frac{1}{2} R_3 \to R_3 \end{array} \qquad \left[\begin{array}{rrrr|r} 1 \amp 0 \amp -2 \amp 0 \amp 0 \\ 0 \amp 1 \amp 1 \amp -1 \amp 0 \\ 0 \amp 0 \amp 1 \amp -1/2 \amp 0 \end{array}\right] \end{equation*}
\begin{equation*} \begin{array}{r} -R_3 + R_2 \to R_2 \\ 2R_3 + R_1 \to R_1 \end{array} \qquad \left[\begin{array}{rrrr|r} 1 \amp 0 \amp 0 \amp -1 \amp 0 \\ 0 \amp 1 \amp 0 \amp -1/2 \amp 0 \\ 0 \amp 0 \amp 1 \amp -1/2 \amp 0 \end{array}\right] \end{equation*}
Rewriting as equations results in
\begin{align*} k_1 -k_4 \amp = 0 \\ k_2 -\frac{1}{2}k_4 \amp = 0 \\ k_3-\frac{1}{2}k_4 \amp = 0 \end{align*}
or solving for the leading variables \(k_1, k_2, k_3\)
\begin{align*} k_1 \amp =k_4 \\ k_2 \amp =\frac{1}{2}k_4\\ k_3 \amp = \frac{1}{2}k_4 \end{align*}
Since \(k_1, k_2\) and \(k_3\) all depend on \(k_4\text{,}\) a free variable, there are an infinite number of solutions to this linear system. We can write this as
\begin{equation*} \{ (k_4, \frac{1}{2} k_4, \frac{1}{2} k_4, k_4) \; | \; k_4 \in \mathbb{R} \}, \end{equation*}
Solutions include \((0,0,0,0)\text{,}\) \((2,1,1,2)\text{,}\) \((-5,-5/2,-5/2,-5)\text{.}\)
Since the above example comes from chemical equations, it is desirable to find a solution of all positive integers, since the number represent the number of atoms in the reaction. In addition, typically the solution with smallest integers is desired. In this case, the solution \((2,1,1,2)\) results in the following equation:
\begin{equation*} 2 NH_3 + H_2O_2 \rightarrow N_2 H_4 + 2 H_2 O \end{equation*}

Subsection 1.3.5 Row Equivalence

We have used the term matrix reduction in converting a matrix to either echelon or reduced row echelon form.

Definition 1.3.15.

If matrix \(A\) can be converted to matrix \(B\) by elementary row operations, then we say that \(A\) and \(B\) are row equivalent or just equivalent. We use the notation \(A \sim B\) for equivalence of matrices.

Example 1.3.16.

Show that the following matrices are row equivalent
\begin{align*} \begin{bmatrix}3 \amp 4 \\ 0 \amp -2 \end{bmatrix} \amp \amp \begin{bmatrix}1 \amp 2\\ 0 \amp -2 \end{bmatrix} \amp \amp \begin{bmatrix}1 \amp 0\\ 0 \amp -2 \end{bmatrix} \end{align*}
Solution.
One method to showing matrices are equivalent is to find the row operation to go from one to the other. However, generally an easier way is to put both (or in this case all three) in reduced row echelon form. In this case all three of these matrices reduce to
\begin{equation*} \begin{bmatrix} 1 \amp 0 \\ 0 \amp 1 \end{bmatrix} \end{equation*}

Proof.

Consider the \(m\) linear combinations \(c_{1,1} x_1 + \cdots + c_{1,n} x_n\) through \(c_{m,1} x_1 + \cdots + c_{m,n} x_n\text{.}\) A linear combination of these is
\begin{align*} \amp d_1 (c_{1,1} x_1 + \cdots + c_{1,n} x_n ) + \cdots + d_m (c_{m,1} x_1 + \cdots + c_{m,n} x_n)\\ \amp \qquad = d_1 c_{1,1} x_1 + \cdots + d_1 c_{1,n} x_n + \cdots + d_m c_{m,1} x_1 + \cdots + d_m c_{m,n} x_n \\ \amp \qquad = (d_1 c_{1,1} + \cdots + d_m c_{m,1}) x_1 + \cdots + (d_1 c_{1,n} + \cdots + d_m c_{m,n}) x_n \end{align*}
We will use the following standard notation for matrices. A matrix is named with a roman capital letter, the \(i\)th row will be denoted with the equivalent lower-case greek with subscript \(i\text{,}\) that is
\begin{align*} A \amp = \begin{bmatrix} \cdots \amp \alpha_1 \amp \cdots \\ \cdots \amp \alpha_2 \amp \cdots \\ \amp \vdots \amp \\ \cdots \amp \alpha_n \amp \cdots \\ \end{bmatrix} \amp B \amp = \begin{bmatrix} \cdots \amp \beta_1 \amp \cdots \\ \cdots \amp \beta_2 \amp \cdots \\ \amp \vdots \amp \\ \cdots \amp \beta_n \amp \cdots \\ \end{bmatrix} \end{align*}

Proof.

The base step, we prove that zero row operations that transforms \(A\) to \(B\text{,}\) satisfies the corollary. In this case,
\begin{equation*} \vec{\beta}_i = 0 \cdot \alpha_1+ 0 \alpha_2 + \cdots + 1 \cdot \alpha_i + \cdots + 0 \cdot \alpha_n \qquad \text{for each} \;i=1, 2, \ldots, n. \end{equation*}
so each row of \(B\) is a linear combination of \(A\text{.}\)
For the inductive step, assume that transforming \(A\) to \(B\) in \(t+1\) steps. Also assume that the matrix in the step before \(B\) is called \(G\text{,}\) or
\begin{equation*} A \rightarrow \cdots \rightarrow G \rightarrow B \end{equation*}
with each arrow denoting a row operation. Now we proceed showing that each of the 3 elementary row operations are linear combinations. If \(G \rightarrow B\) is given by
  1. \(c R_i \rightarrow R_i\text{,}\) then
    \begin{equation*} \vec{\beta}_j = \begin{cases} c \vec{\gamma}_j \amp \text{if]}\;\; i=j \\ \vec{\gamma}_j \amp \text{otherwise} \end{cases} \end{equation*}
    each of which is a linear combination of the rows of \(G\text{,}\)
  2. \(R_i \leftrightarrow R_j\text{,}\) for \(i \neq j\text{,}\) then
    \begin{equation*} \vec{\beta}_k = \begin{cases} \vec{\gamma}_i \amp \text{if}\;\; k=j \\ \vec{\gamma}_j \amp \text{if}\;\; k=i \\ \vec{\gamma}_k \amp \text{otherwise} \end{cases} \end{equation*}
    again, each of which is a linear combination of the rows of \(G\text{.}\)
  3. \(c R_i + R_j \rightarrow R_j\) for \(i \neq j\text{,}\) then
    \begin{equation*} \vec{\beta}_k = \begin{cases} c \vec{\gamma}_k+ \vec{\gamma}_j \amp \text{if}\;\; k=i \\ \vec{\gamma}_k \amp \text{otherwise} \end{cases} \end{equation*}
    again, each of which is a linear combination of the rows of \(G\text{.}\)
Now this proves that for the step \(G \rightarrow B\text{,}\) each row of \(B\) is a linear combination of rows of \(G\text{.}\)
Now, using induction, since the first \(t\) steps transforming \(A\) to \(G\) is a linear combination, the steps from \(A\) to \(B\text{,}\) each row of \(B\) is a linear combination of rows of \(A\) because linear combinations of linear combinations are linear combinations.

Example 1.3.19.

Show that the steps that transform:
\begin{align*} \amp\begin{bmatrix} 0 \amp 3 \\ 1 \amp 2 \end{bmatrix}\\ R_1 \leftrightarrow R_2 \qquad \amp \begin{bmatrix} 1 \amp 2 \\ 0 \amp 3 \\ \end{bmatrix}\\ \frac{1}{3} R_2 \rightarrow R_2 \qquad \amp \begin{bmatrix} 1 \amp 2 \\ 0 \amp 1 \\ \end{bmatrix}\\ -2 R_2 + R_1 \rightarrow R_1 \qquad \amp \begin{bmatrix} 1 \amp 0 \\ 0 \amp 1 \end{bmatrix} \end{align*}
are each linear combinations of row operations.
Solution.
Call the 4 matrices, \(A\text{,}\) \(D\text{,}\) \(G\) and \(B\text{,}\) then
\begin{align*} \begin{split} \delta_1 \amp = \alpha_2 \\ \delta_2 \amp = \alpha_1 \end{split} \amp\amp\amp \begin{split} \gamma_1 \amp = \alpha_1 \\ \gamma_2 \amp = \frac{1}{3} \alpha_2 \\ \end{split} \amp\amp\amp \begin{split} \beta_1 \amp = -2 \gamma_2 + \gamma_1 \\ \beta_2 \amp = \gamma_2 \end{split} \end{align*}

Subsection 1.3.6 Echelon Form and Linear Combinations of Rows

Let’s return to echelon form. Why is this a nice form? Well we can use back substitution to solve for the leading variables. The reason this works is the following lemma.
In lieu of a proof, let’s take a look at an example. Consider the echelon form matrix of a the linear system from ExampleΒ 1.4.5.
\begin{equation*} \left[\begin{array}{rrrrr|r} 0 \amp 1 \amp 3 \amp -9 \amp 11 \amp 14 \\ 0 \amp 0 \amp 2 \amp 0 \amp 4 \amp 9 \\ 0 \amp 0 \amp 0 \amp 0 \amp 3 \amp 27 \\ \end{array}\right] \end{equation*}
We look and determine if the second row (as an example) is a linear combination of the other two rows. Thus:
\begin{align*} \begin{bmatrix} 0 \amp 0 \amp 2 \amp 0 \amp 4 \amp 9 \\ \end{bmatrix} \amp = c_1 \begin{bmatrix} 0 \amp 1 \amp 3 \amp -9 \amp 11 \amp 14 \end{bmatrix} + c_2 \begin{bmatrix} 0 \amp 0 \amp 0 \amp 0 \amp 3 \amp 27 \end{bmatrix} \end{align*}
Each component then becomes an equation for the linear system:
\begin{align*} 0 c_1 + 0 c_2 \amp = 0 \\ c_1 + 0 c_2 \amp = 0 \\ 3 c_1 + 0 c_2 \amp = 2 \\ -9 c_1 + 0 c_2 \amp = 0 \\ 11 c_1 + 3 c_2 \amp = 4 \\ 14 c_1 + 27 c_2 \amp = 9 \end{align*}
which can be written as the augmented matrix:
\begin{equation*} \left[\begin{array}{rr|r} 0 \amp 0 \amp 0 \\ 1 \amp 0 \amp 0 \\ 3 \amp 0 \amp 2 \\ -9 \amp 0 \amp 0 \\ 11 \amp 3 \amp 4 \\ 14 \amp 27 \amp 9\\ \end{array}\right] \end{equation*}
if we transform this to row echelon form we get:
\begin{equation*} \left[\begin{array}{rr|r} 14 \amp 27 \amp 9\\ 0 \amp -3 \amp -1\\ 0 \amp 0 \amp 28\\ 0 \amp 0 \amp 0\\ 0 \amp 0 \amp 0\\ 0 \amp 0 \amp 0\\ \end{array}\right] \end{equation*}
and the 3rd row says ``no solution’’, so that the 2nd row is not a linear combination of the other two rows.
The notation of rows not being linear combinations of other rows is called independence and will be defined more-explicitly later in the course, however this example should give you a sense of what independence means.

Proof of LemmaΒ 1.3.20.

This will be a proof by contradiction. Assume that the \(i\)th row is a linear combination of the other rows of the matrix that is it can be written:
\begin{equation} \alpha_i = c_1 \alpha_1 + c_2 \alpha_2 + \cdots + c_{i-1} \alpha_{i-1} + c_{i+1} \alpha_{i+1} + \cdots + c_m \alpha_m\tag{1.3.2} \end{equation}
The matrix \(A\) can be written as
\begin{equation*} A = \begin{bmatrix} 0 \amp \cdots \amp 0 \amp \alpha_{1,\ell_1} \amp \alpha_{1, \ell_1+1} \amp \cdots \amp \alpha_{1,\ell_i} \amp \cdots \amp \alpha_{1,\ell_m} \\ 0 \amp \cdots \amp 0 \amp 0 \amp \alpha_{2,\ell_2} \amp \cdots \amp \alpha_{2,\ell_i} \amp \cdots \amp \alpha_{2, \ell_m} \\ \amp \amp \amp \amp \amp \amp \vdots \amp \amp \amp \vdots \\ \vdots \amp \amp \vdots \amp \amp 0 \amp \amp \alpha_{i,\ell_i} \amp \amp \vdots \\ 0 \amp \cdots \amp 0 \amp 0 \amp 0 \amp 0 \amp 0 \amp \\ 0 \amp \cdots \amp \amp \amp \amp \amp0 \amp0 \amp \alpha_{m,\ell_m} \end{bmatrix} \end{equation*}
To find the values of \(c_j\) in (\ref{eq:row:linear:comb}), evaluate that equation for any column. To begin, consider \(\ell_1\text{,}\) the column of the leading coefficient of the first row. Because the matrix is in echelon form, \(\alpha_{2,\ell_1}, \alpha_{3,\ell_1}, \ldots, \alpha_{m,\ell_1}\) are all zero. Thus, examining the \(\ell_1\)th element of (1.3.2), one gets:
\begin{equation*} 0 = c_1 \alpha_{1,\ell_1} + c_2 \cdot 0 + \cdots + c_m \cdot 0 \end{equation*}
and because \(\alpha_{1,\ell_1}\neq 0\) (it is a leading element), \(c_1=0\text{.}\)
Inductively, assume that \(c_1,c_2, \ldots, c_{k-1}=0\text{.}\) Let \(\ell_k\) be the column with the leading element in row \(k\text{.}\) This means that \(\alpha_{k+1,\ell_k}, \alpha_{k+2,\ell_k}, \ldots, \alpha_{m,\ell_k}\) are all zero. If we examine the \(\ell_k\)th component of (1.3.2), then
Let \(\ell_i\) be the column with the leading element in row \(i\text{.}\) If the \(\ell_i\)th element of (\ref{eq:row:linear:comb}) is extracted, one gets
\begin{align*} \alpha_{i,\ell_k} \amp = (0) \alpha_{1,\ell_k} + (0) \alpha_{2,\ell_k} + \cdots + (0) \alpha_{i-1,\ell_k} + (0) \alpha_{i+1,\ell_k} \\ \amp \qquad + \cdots + c_k\alpha_{k,\ell_k} + c_{k+1} \cdot 0 + \cdots + c_m \cdot 0 \end{align*}
If \(i>k\text{,}\) then \(\alpha_{i,\ell_k}=0\text{,}\) and this implied that \(c_k=0\text{.}\) If \(i \lt k\text{,}\) then