Lecture 7

A. Agarwal
February 14, 2012

Recall

  1. 1.

    A set of vectors {v1,v2,,vk} in n is linearly independent if the following homogenous equation has only the zero (trivial) solution:

    c1v1+c2v2++ckvk =0 (1)

    Note that (1) always has a solution, namely c1=c2==ck=0. Our interest is whether this is the only solution, or if there are others.

    It is a common mistake when we first learn this to think that something like “since we can form a linear combination with c1,c2,,ck=0, thus the set is dependent.”. This is not the case; it must be a solution with at least one coefficient non-zero.

  2. 2.

    If the number of vectors in S is greater than the dimension of the space, then S is dependent.

    Suppose {v1,v2,,vk} is a dependent set. Then there is at least one ci0. Without loss of generality, say c10. Then

    c1v1 =-c2v2--ckvk
    v1 =-c2c1v2--ckc1vk

    The righthand side of this equation should remind us of the span. In this case, of Span{v2,,vk}. That is, v1 lies in the span of the other vi’s.

Linear Independence

Question:

Consider the set of vectors S={0,v2,,vk}. Is S independent or dependent? To answer this, we must ask whether the following homogeneous equation has a non-zero solution:

c10+c2v2++ckvk =0

Consider for instance the solution c1=3, c2,c3,,ck=0. This is a non-zero or non-trivial solution, hence S must be dependent.

Question:

Let S={v1,v2,v3}. Suppose none of the vectors in S are zero vectors, and each is perpendicular to one another. That is,

vi0 vivjfor all ij

Is S independent or dependent? Before we answer this question, we should observe some noteworthy aspects of this question and how it is posed. If we can find some useful relationships between orthogonality and linear dependence, we will perhaps uncover illuminating geometric relationships.

To answer the question at hand, we can start with the equation we already know: the linear combination of vectors that equals the zero vector:

c1v1+c2v2+c3v3 =0

it is initially unclear how the orthogonality will come into play, but consider taking the dot product of both sides of the equation with v1:

(c1v1+c2v2+c3v3)v1 =0v1
c1(v1v1)+c2(v2v1)+c3(v3v1) =0 Distributive property of dot product
c1||v1||2+c2(0)+c3(0) =0 The vectors are orthogonal to each other
c1||v1||2 =0

Since v10, we know that ||v1||20. Thus from the above, c1=0.

Likewise, we find that c2=c3=0 by taking the dot product with v2 and v3.

Therefore, the only solution is the zero solution. Thus S must be independent.

Note: In an n-dimensional space we cannot have more than n (non-zero) orthogonal vectors.

Matrices

We have already seen some examples of matrices as means of storing systems of linear equations. They are simply arrays of numbers laid out in a way in which each entry has a convenient address.

𝐀=[a1,1a1,2a1,3a2,1a2,2a2,3]  𝐀2×3

The i-jth entry is the entry found in row i and column j. We always give the address of the entry row first and column second

Every matrix is described by its entries, and we sometimes write it like below:

𝐀=(aij)  aij,

For our purposes, the entries will almost always be from and , but keep in mind that the entries could be any object.

It is also important to note the size of a matrix, and we often do this by saying that 𝐀 is an n×m matrix, or writing 𝐀n×m.

Algebra of Matrices

Suppose 𝐀m×n and 𝐁m×n are matrices of the same size. We define several operations on matrices as follows:

Operation Definition by Entry Resultant Size
Matrix addition 𝐀+𝐁 = (aij+bij) m×n
Matrix subtraction 𝐀-𝐁 = (aij-bij) m×n
Scalar multiplication c𝐀 = (caij)  c is a scalar m×n

Example.

𝐀=[12301-5] 𝐁=[56-6003] 𝐂=[1230]

Then 𝐀+5𝐁 is given by

𝐀+5𝐁=[12301-5]+[2530-300015]=[2632-270110]

Some may wonder what the result of an addition such as 𝐀+𝐂 would be. Since the dimensions of these matrices do not match, such an addition is not defined. It is common to wonder whether we could just add a column of zeros to the matrix 𝐂 in order to make the operation defined. The response to this is that of course we could do that, but it would completely change the matrix 𝐂, so we do not allow this type of operation.

These are the simplest types of operations we can define on matrices. Two elementary operations remain: multiplication and division. We might also wonder about operations such as logarithm and exponentiation. We will get to those types of operations later.

Matrix Multiplication

Matrix multiplication at first seems very strange. If we consider trying to define a multiplication on two matrices, we might conclude from seeing addition and subtraction that to multiply matrices they should be the same size. In fact this is not the case.

Before we discuss the full definition of matrix multiplication, consider the dot product:

[123][abc]=a+2b+3c

If the vectors had an unequal number of components then the dot product would not be defined. If we consider the scalar a+2b+3c to be the sole entry in a matrix of size 1×1, then this tells us one possible set of sizes for which matrix multiplication is defined:

[123]1×3[abc]3×1=[a+2b+3c]1×1

The dimensions of matrices that can be multiplied can be summarized as

𝐀m×n𝐁n×p=𝐂m×p

Example.

𝐀2×3=[123-140] 𝐁3×1=[pqr]

Consider the product

𝐀𝐁 =[123-140][pqr]

We can evaluate this by ignoring one row at a time of 𝐀:

𝐀𝐁=[123][pqr]=[p+2q+3r]𝐀𝐁=[-140][pqr]=[-p+4q+0r]} 𝐀𝐁=[123-140][pqr]=[p+2q+3r-p+4q+0r]2×1

It may be startling to observe the outcome of trying to multiply 𝐁 by 𝐀:

𝐁3×1𝐀2×3 =Not defined

Because the dimensions of the two matrices do not correctly line up in this order, they cannot be multiplied in this way. The technical term for this is that, in general, matrix multiplication is non-commutative. That is, 𝐀𝐁𝐁𝐀. In fact, both need not be defined at all, depending on their sizes.

Another Perspective on Matrix Multiplication

Consider again the matrix multiplication from the previous example. We can view this multiplication in a number of ways:

𝐀𝐁 =[123-140][pqr]
=[p+2q+3r-p+4q+0r] Each row is a linear combination of rows of 𝐁
=p[1-1]+q[24]+r[30] Linear combination of the columns of 𝐀

The three ways of viewing matrix multiplication are

  1. 1.
    cij=(row i of 𝐀)(col. jof 𝐁)
  2. 2.

    ith row of 𝐂 is a linear combination of rows of 𝐁 with coefficients from the ith row of 𝐀.

  3. 3.

    jth column of 𝐂 is a linear combination of columns of 𝐀 with coefficients from the jth column of 𝐁.

Example.

We will try multiplication with each of the three ways of viewing matrix multiplication.

𝐀=[1201] 𝐁=[1200-112]

Note that 𝐀𝐁 is not defined, but 𝐁𝐀 is defined. Let 𝐂=𝐁𝐀, the matrix whose entries we will try to find.

  1. 1.

    First method:

    𝐂=[c11c12c21c22c31c32]3×2
    c11 =(12)Row 1 of 𝐁(10)Column 1 of 𝐀=1

    Similarly,

    c32 =(-112)(21)=10

    So

    𝐂=[1c12c21c22c3110]3×2
  2. 2.

    Next, we will use the second method. This says that rows of 𝐂 are linear combinations of rows of 𝐀.

    Row 1 =1[12]+2[01]

    The coefficients 1 and 2 in front of the rows of 𝐀 are taken from the first row of 𝐁.

    𝐂=[14c21c22c3110]3×2
  3. 3.

    The next method states that columns of 𝐂 are linear combinations of columns of 𝐁.

    Column 2 =2[10-1]+1[2012]
    =[4010]

    This allows us to fill in another entry of 𝐂:

    𝐂=[14c210c3110]3×2