Previous Chapter

Return to all notes

Next chapter

Linear Algebra in a huge field of mathematics and many standard operations in linear algebra is Matlab. The Mat in Matlab is for matrix, so linear algebra is at the heart of Matlab.

Vectors

There are two standard types of Vectors, column vectors and row vectors and Matlab has both. If you want to enter the column vector \(\left[\begin{array}{c} 4 \\ -2 \\ 3 \end{array} \right]\)

you can type:

v=[4;-2;3]

If you want a row vector, like \(\left[\begin{array}{ccc} 1 & 2 & 3 \end{array}\right]\) you can enter this as

v2 = [1,2,3]

or [1 2 3]. Recall also, the colon operator. This will generate row vectors. For example, v3=1:4 is the row vector \(\left[\begin{array}{cccc} 1 & 2 & 3 & 4 \end{array}\right]\)

We can use the transpose operator ' along with the colon notation to get column vectors:

w = (1:3:7)'

generates the column vector: \(\left[\begin{array}{c} 1 \\ 4 \\ 7 \end{array} \right]\)

We will see the tranpose operator on arrays below, but if you apply it to vectors, it switches between row vectors and column vectors.

Dot and Cross Products

You can take the dot product of two column vectors of the same length with the command dot. For example

dot(v,w)

returns 17.

and to find the cross product, try

cross(v,w)

which returns the vector \(\left[\begin{array}{r} -26 \\ -25 \\ 18 \end{array}\right]\)

Matrices

To enter a matrix in Matlab, it is entered row by row. Consider the matrix: \(A=\left[\begin{array}{cccc} 1 & 2 & 3 & 4 \\ 5 & 6 & 7 & 8 \\ 9 & 10 & 11 & 12 \end{array} \right]\) you can enter this as

A=[1,2,3,4;5,6,7,8;9,10,11,12]

Let’s also define the matrix $B$ to be a 3 by 4 matrix of all ones, which can be done with

B=ones(3,4)

and this will look like: \(\left[\begin{array}{cccc} 1 & 1 & 1 & 1 \\ 1 & 1 & 1 & 1 \\ 1 & 1 & 1 & 1 \end{array} \right]\)

And a 3 by 3 matrix with the numbers 2, $-1$ and 4 on the diagonal,

C=diag([2,-1,4])

and this will look like \(\left[\begin{array}{cccc} 2 & 0 & 0 \\ 0 & -1 & 0 \\ 0 & 0 & 4 \end{array} \right]\)

Adding and Subtracting Matrices

We can add the matrices $A$ and $B$ because they are the same size and can using standard notation:

A+B

which will return: \(\left[\begin{array}{cccc} 2 & 3 & 4 & 5 \\ 6 & 7 & 8 & 9 \\ 10 & 11 & 12 & 13 \end{array} \right]\)

and if we subtract

A-B

Matlab returns \(\left[\begin{array}{cccc} 0 & 1 & 2 & 3 \\ 4 & 5 & 6 & 7 \\ 8 & 9 & 10 & 11 \end{array} \right]\)

If we try to add $A$ and $C$, we get the error:

Matrix dimensions must agree.

this is because the two matrices are not the same size.

Scalar multiplication

Recall that Scalar multiplication of either a vector or a matrix results in a vector a matrix that is the product of the scalar and each element of the vector or matrix. For example, $3v$ for the vector $v$ above is \(\left[\begin{array}{c} 8 \\ -4 \\ 3\end{array}\right]\) and $-2A$ is

\[\left[ \begin {array}{cccc} -2&-4&-6&-8\\ -10&-12&-14&-16\\ -18&-20&-22&-24 \end {array} \right]\]

Multiplying Matrices

Recall that two multiply 2 matrices, the number of columns of the first matrix must equal the number of rows of the second matrix. For example if we have the matrix

P=[3,2,0;1,-2,4;0,2,5]

If we multiply $P$ and $A$, using P*A to get

\[\left[ \begin{array}{cccc} 13&18&23&28\\ 27&30&33& 36\\ 55&62&69&76 \end{array} \right]\]

We can also use the dot notation to multiply matrices and vectors. For example, P*v is

\[\left[\begin{array}{c} 8\\20\\11\end{array}\right]\]

If we don’t have compatibly-sized matrices, for example, $A$ and $B$, and try A*B, we get:

Error using  *
Incorrect dimensions for matrix multiplication. Check that the number
of columns in the first matrix matches the number of rows in the
second matrix. To perform elementwise multiplication, use '.*'.

Exercise

Let

\(A=\left[\begin{array}{cc} 1 & 3 \\ -2 & 4 \\ 5 & 0\end{array}\right]\) \(B=\left[\begin{array}{ccc} 3 & 2 & 3 \\ 0 & 2 & 0 \\ -1 &4 & -1 \end{array}\right]\) \(C=\left[\begin{array}{cc} 0 &4 \\ -2 & 1 \end{array}\right]\) \(\vec{v} = \left[\begin{array}{c} 4 \\ 2 \end{array}\right]\) \(\vec{u} = \left[\begin{array}{c} 0 \\ 5 \\ -2 \end{array}\right]\)

Find the following or state the operation is not compatible.

1. $AB$
2. $AC$
3. $BC$
4. $A\vec{v}$
5. $A\vec{u}$
6. $B\vec{u}$.

Identity Matrix

A square matrix with all zeros except 1’s on the diagonal, is called an identity matrix. The identity matrices of size 2 by 2, 3 by 3 and 4 by 4 are

\[\begin{array}{ccc} \left[\begin{array}{cc}1&0\\0&1\end{array}\right]&\left[\begin{array}{ccc}1&0&0\\0&1&0\\0&0&1\end{array}\right]& \left[\begin{array}{cccc}1&0&0&0\\0&1&0&0\\0&0&1&0\\0&0&0&1\end{array}\right]\end{array}\]

An identity matrix has the property that if you mulitply a matrix or a vector by the identity matrix, the matrix or vector is returned. More particularly, \(\begin{array}{cccc}IA=A & AI=A & I\vec{v}=\vec{v} & \vec{v}I=\vec{v}\end{array}\)

To get an Identity matrix in Matlab, use the eye command. For example

eye(3)

returns a 3 by 3 identity matrix.

Exercise: Matrix Multiplication of Identity Matrices

Using the vector $\vec{v}$ and matrix $A$ above, find

1. $I\vec{v}$
2. $\vec{v}I$
3. $IA$
4. $AI$.

and show that you get the vector or matrix back. Make sure that you use the identity matrix of the correct size.

Matrix Transpose

Recall that the transpose of a matrix is a new matrix where the rows of the tranpose are the columns of the original matrix. For example,

\[A^{T} = \left[\begin{array}{ccc} 1 & 5 & 9 \\ 2 & 6 & 10 \\ 3 & 7 & 11 \\ 4 & 8 & 12 \end{array}\right]\]

This can be done in Matlab using the ' command:

A'

returns the matrix above.

Gaussian Elimination

One of the first concepts in linear algebra is how to solve linear systems. For example, \(\begin{array}{rl} 3x- 2y & = 4 \\ x + 5y & = 7 \end{array}\)

Matlab can solve this by using the solve command such as

syms x y
[x_ans,y_ans]=solve([3*x-2*y==4,x+5*y==7])

which returns ${x=2,y=1}$. In linear algebra, we find the augmented coefficient matrix as \(\left[\begin{array}{ccc} 3 & -2 & 4 \\ 1 & 5 & 7 \end{array}\right]\)

One method is to use row operations on the matix to reduce the matrix to a more useful form. We can put the matrix into Reduced-Row Echelon Form with

rref(A)

which returns:

\(\left[\begin{array}{ccc}1 & 0 & 2\\0 &1 &1\end{array}\right]\) and this can be written as the two equations $x=2$, $y=1$, same as above.

Another example is

A=[ ...
    5 3 5 1; ...
    5 4 1 1 ; ...
    5 2 2 5]

where the … means to continue the input. This is used to make it easier to read as a matrix. If we do rref(A), then the result is:

    1.0000         0         0    2.1429
         0    1.0000         0   -2.2857
         0         0    1.0000   -0.5714

and perhaps we want fractions instead of decimals–to ensure no round off errors. To use exact arithmetic, we’ll make a symbolic version of the matrix with

A= sym([ ...
    5 3 5 1; ...
    5 4 1 1 ; ...
    5 2 2 5])

and then row-reducing with

rref(A)

results in \(\left(\begin{array}{cccc} 1 & 0 & 0 & \frac{15}{7}\\ 0 & 1 & 0 & -\frac{16}{7}\\ 0 & 0 & 1 & -\frac{4}{7} \end{array}\right)\)

This shows the solution, $x=15/7, y=-16/7, z=-4/7$.

Exercise: Row Reduced Echelon Form

The linear system \(\begin{array}{rl}-3x+4y-5z & = 8\\ -4x-2y-z & = 0 \\ 5x-4y+3z & = -16\end{array}\) can be written as an Augmented Coefficient Matrix: \(\left[\begin{array}{cccc}-3 & 4 & -5 & 8\\ -4&-2&-1&0\\5&-4&3&-16\end{array}\right]\)

Use rref to reduce the matrix and then write the solution.

Determinants

The determinant of a matrix is helpful in many different matrix applications including determining if the matrix is singular and it can be used in solving linear systems using Cramer’s Rule.

The determinant is applied only to square matrices and is denoted ${\rm det}(A)$ or $|A|$. If the matrix is a 2 by 2, then \(\left\vert\begin{array}{cc} a & b \\ c & d \end{array} \right\vert=ad-bc\)

When finding a determinant, often Laplace’s Expansion method or row operations are used to find the value. For example

\[\left\vert\begin {array}{ccc} 2&-3&1\\ 3&3&0\\2&3&-3\end {array} \right\vert=2\left\vert\begin{array}{cc}3&0\\3&-3\end{array}\right\vert-(-3)\left\vert\begin{array}{cc}3&0\\2&-3\end{array}\right\vert+1\left\vert\begin{array}{cc}3&3\\2&3\end{array}\right\vert\] \[\begin{array}{rl}&=2\left((3)(-3)-(3)(0)\right)+3\left((3)(-3)-2(0)\right)+\left((3)(3)-2(3)\right)\\ &=2(-9)+3(-9)+(9-6)=-42\end{array}\]

And using

det(A)

returns $-42$ as well.

Matrix Inverse

If a matrix, $A$ is square and it’s determiant is not zero, then there exists a matrix called the inverse and denoted with the property $AA^{-1}=I$,

One way to find the matrix inverse of $A$ is write the matrix augmented with the identity matrix and then put it in Reduced Row echelon form.

For example, consider the matrix

\(A=\left[\begin{array}{ccc}-3&4&-5\\-4&-2&-1\\5&-4&3\end{array}\right]\) then we write the augmented matrix where the 3 by 3 identity matrix is append on the right side. We can to this using the command:

aug=cat(2,A,eye(3))

where the 2 means to concatenate by colums (dimension 2), the two matrices A and eye(3), 3 by 3 identity matrix. This returns \(\left[\begin{array}{ccc}-3&4&-5&1&0&0\\-4&-2&-1&0&1&0\\5&-4&3&0&0&1\end{array}\right]\) We can then use rref(aug) to get

\[\left[ \begin {array}{cccccc} 1&0&0&{\frac {5}{36}}&-\frac{1}{9}&{\frac {7}{ 36}}\\ 0&1&0&-{\frac {7}{72}}&-\frac{2}{9}&-{\frac {17}{72}} \\ 0&0&1&-{\frac {13}{36}}&-\frac{1}{9}&-{\frac {11}{36}} \end {array} \right]\]

and the right half of the matrix is the inverse or \(A^{-1}=\left[ \begin {array}{ccc} {\frac {5}{36}}&-\frac{1}{9}&{\frac {7}{ 36}}\\ -{\frac {7}{72}}&-\frac{2}{9}&-{\frac {17}{72}} \\ -{\frac {13}{36}}&-\frac{1}{9}&-{\frac {11}{36}} \end {array} \right]\)

We can also get the same result from the command inv:

inv(sym(A))

returns the same matrix above.

Exercise: Finding Inverse Matrices

1. Find the inverse of \(C=\left[ \begin {array}{ccc} -1&-3&0\\-3&2&-1\\1&-3&0\end {array} \right]\) using the augmented matrix with the identity matrix.
2. Find the inverse of $C$ using the command inv
3. What happens when you try to find the inverse of \(\left[ \begin {array}{ccc} -1&-3&0\\-3&2&-1\\2&-5&1\end{array}\right]?\)

Using the Matrix Inverse to solve a linear system

Above we saw the linear system:

The linear system \(\begin{array}{rl}-3x+4y-5z & = 8\\ -4x-2y-z & = 0 \\ 5x-4y+3z & = -16\end{array}\)

and used the Augumented Matrix to find the solution. Instead, we can write: \(\begin{array}{rl}A=\left[ \begin {array}{ccc} -3&4&-5\\-4&-2&-1 \\5&-4&3\end {array} \right]&\vec{b}=\left[\begin{array}{c}8\\0\\-16\end{array}\right]\end{array}\)

and then if we write the vector \(\vec{x}=\left[\begin{array}{c}x\\ y\\ z\end{array}\right],\) then the linear system can be written as $A\vec{x}=\vec{b}$. We can solve this in the following way. \(A\vec{x}=\vec{b}\) Multiply the equation by $A^{-1}$ \(\begin{array}{rl}A^{-1}A\vec{x}&=A^{-1}\vec{b}\qquad \text{since $A^{-1}A=I\qquad\qquad$}&\\ I\vec{x}&=A^{-1}\vec{b}\qquad \text{since $I\vec{x}=\vec{x}\qquad\qquad$}&\\ \vec{x}&=A^{-1}\vec{b}\end{array}\)

Use this to solve the linear system above.

First, we will find $A^{-1}$ using the inv command:

Ainv=inv(sym(A))

which returns \(\left[ \begin {array}{ccc} {\frac {5}{36}}&-\frac{1}{9}&{\frac {7}{ 36}}\\ -{\frac {7}{72}}&-\frac{2}{9}&-{\frac {17}{72}} \\ -{\frac {13}{36}}&-\frac{1}{9}&-{\frac {11}{36}} \end {array} \right]\) as we saw above.

Lastly, we multiply $A^{-1}$ and $\vec{b}$

Ainv*b

to get: \(\left[\begin{array}{c}-2\\3\\2\end{array}\right]\)

so the result is $x=-2,y=3,z=2$, the same as we found above.

Eigenvalues and Eigenvectors

Another important topic in Linear Algebra is that of eigenvalues and eigenvectors. Recall that a scalar (number) $\lambda$ is an eigenvalue of a square matrix $A$ with corresponding eigenvector $\vec{v}$ if this satisfies the equation \(A\vec{v}=\lambda\vec{v}.\) Finding $\lambda$ and $\vec{v}$ that satisfy this require two steps:

1. The eigenvalues can be found by solving \(\vert A-\lambda I\vert=0\)where the vertical lines is the determinant of the matrix.

2. For each eigenvalue, you seek for a vector $\vec{v}$ such that \((A-I\lambda)\vec{v}=\vec{0}\)

Example: Eigenvalues and Eigenvectors

Find the eigenvalues and eigenvectors of the matrix \(A=\left[\begin{array}{cc}4&-1\\-2&3\end{array}\right]\)

Step 1: Solve $\vert A-\lambda I\vert=0$, the characteristic equation of the matrix $A$.

syms s
A = [4 -1; -2 3]
c_eqn = det(A-s*eye(2))

returns $s^{2}-7s+10=0$. Matlab doesn’t have the ability to use $\lambda$ as a vraiable.

eigs = solve(c_eqn==0)

returns $2,5$. There are two eigenvalues for this. Note: the maximum number of eigenvalues for an $n$ by $n$ matrix is $n$.

Step 2: For each of the eigenvalues, solve $(A-\lambda I)\vec{v}=0$. First, consider $\lambda=2$

A-eigs(1)*eye(2)

returns \(\left[\begin{array}{cc} 2 & -1\\ -2 & 1 \end{array}\right]\) The eigenvector is a solution to $(A-\lambda I)\vec{v} = \vec{0}$. We can find $\vec{v}$ by row-reducing the augmented matrix found by

aug = cat(2,A-eigs(1)*eye(2),zeros(2,1))

which is the matrix: \(\left[\begin{array}{ccc} 2 & -1 & 0\\ -2 & 1 & 0 \end{array}\right]\) and row-reducing this with

rref(aug)

returns: \(\left[\begin{array}{ccc} 1 & -\frac{1}{2} & 0\\ 0 & 0 & 0 \end{array}\right]\) If we use variables $x$ and $y$ with this system, the first row of the matrix becomes: \(x -\frac{1}{2}y = 0\) or \(x = \frac{1}{2}y\) and if we select $y=2$, then $x=1$ and then the vector \(\vec{v}_1 = \left[\begin{array}{c} 1 \\ 2 \end{array}\right]\) is the eigenvector.

The eigenvector associated with $\lambda = 5$ is

return the matrix: \(\left[\begin{array}{cc}-1&-1\\-2&-2\end{array}\right]\). To solve this we will append the zero vector on the right or

aug2 = cat(2,A-eigs(2)*eye(2),zeros(2,1))

which returns \(\left[\begin{array}{ccc}-1&-1&0\\-2&-2&0\end{array}\right]\) and then

rref(aug2)

which returns \(\left[\begin{array}{ccc}1&1&0\\0&0&0\end{array}\right]\)

The first row of the equation, means that $x+y=0$. We seek any nonzero solution to this, if $y=-1$ and $x=1$, then the vector is \(\vec{v}_2=\left[\begin{array}{c}1\\-1\end{array}\right]\).

In summary for the matrix \(A=\left[\begin{array}{cc}4&-1\\-2&3\end{array}\right]\)

The eigenvalues are 5 and 2 and the associated eigenvectors are \(\begin{array}{rl}\vec{v}_1=\left[\begin{array}{c}1\\-1\end{array}\right]& \vec{v}_2=\left[\begin{array}{c}1\\2\end{array}\right]\end{array}\)

Exercise: Eigenvalues and Eigenvectors

Use the steps above to find the eigenvalues and eigenvectors of \(A=\left[\begin{array}{cc}6&2\\4&4\end{array}\right]\)

Finding the Eigenvalues and Eigenvectors with a single command

Matlab (along with many related software packages) will find the eigenvalues and eigenvectors of a matrix. Technically, they are usually approximations because the general eigenvalue problem is hard to do from a computational point of view as the matrices get large.

Of course, Matlab can just find the eigenvalues and eigenvectors as well.

Returning to the matrix:\(A=\left[\begin{array}{cc}4&-1\\ -2&3\end{array}\right]\) we can let Matlab find the eigenvalues directly with

eig(A)

and you get: \(\left[\begin{array}{c}5\\2\end{array}\right]\) which are the roots of the characteristic polynomial. If you enter:

[V,S] = eig(A)

it returns both the eigenvectors and eigenvalues. You’ll see:

V = 2×2
    0.7071    0.4472
   -0.7071    0.8944

and

S = 2×2
     5     0
     0     2

The columns of the v matrix are the eigenvectors. Recall that any scalar multiple of an eigenvector is an eigenvector. Matlab scales everything so the length of eigenvectors are 1.

The matrix s has the eigenvalues on the diagonal. These are returned as matrices because there are some nice features of eigenvalue/vectors in matrix form.

Previous Chapter

Return to all notes

Next chapter