Section 1.1: Systems of linear equations. Augmented matrix of a linear system. Elementary row operations.

Read Sections 1.3 and 1.4, and skim Sections 2.1 and 2.2.

We showed that the solution set of a homogeneous system with n unknowns is a subspace of ℝⁿ, called the nullspace of A, and that the set of all vectors b such that Ax=b is consistent is a subspace of ℝ^m, called the column space of A. We showed that the general solution to Ax=b may be expressed in terms of the general solution to the associated homogeneous system, Ax=0, along with a particular solution p to Ax=b:

x = p + (t₁*s₁+ …+t_k*s_k).

The purpose of this representation is to show that the general solution of a linear system may be viewed as a translate of a subspace.

We defined linear (in)dependence. The set of columns of a matrix A are linearly independent iff Ax=0 has only the trivial solution. A subset of a linearly independent set is linearly independent. The empty set of vectors is linearly independent. A 1-element set of vectors, {v}, is linearly independent iff v is nonzero. A 2-element set of vectors is linearly independent iff neither vector is a multiple of the other. An n-element set of vectors is linearly independent iff no one is a linear combination of the others. ℝⁿ does not contain a linearly independent set of size greater than n, but does contain a linearly independent set of size m for any m satisfying 0≤m≤n. Quiz 3.

I defined "linear transformation". (Alternative terminology: "vector space homomorphism".) I gave examples and nonexamples. I showed that every linear transformation T:ℝⁿ→ ℝ^m has the form T(x) = Ax for some uniquely determined (m×n) matrix A, which we showed how to calculate. We examined some examples. One nice example (which I think is worth memorizing) is that the counterclockwise rotation of the plane through an angle θ is a linear transformation from ℝ² to ℝ² whose matrix is: .

I explained why a linear transformation T(x) = Ax is 1-1 iff Ax = 0 has only the trivial solution. As class ended, I posed the problem of identifying what property of the matrix A corresponds to the property that the transformation T is onto.

Introduction to Mathematica: I discussed how to get Mathematica and how to start creating a notebook. Here is a good video tutorial series.

I worked through a simple example using Mathematica to balance a chemical equation. I discussed using linear algebra to solve network flow problems, and to study predator-prey dynamics. Quiz 4.

We discussed the solutions to Quiz 4, especially what it means to "define" a term or phrase. Then we discussed the transpose of a matrix, and the laws of arithmetic related to transpose. The most important (or unexpected) law is that (AB)^t=B^tA^t. (Note: (AB)^t≠A^tB^t, usually. In fact, the dimensions of the matrices usually don't match up properly for this to even make sense.) Finally we discussed left, right and 2-sided inverses of a matrix. We showed that if A has a left inverse, then Ax=0 has only the trivial solution. More about this on Friday …

We continued our discussion of inverses, focusing on the geometric interpretation of "A is left invertible" and "A is right invertible". We added to earlier characterizations of left/right invertibility by proving that A is left invertible iff T(x)=Ax preserves independent sets, and A is right invertible iff T(x)=Ax preserves spanning sets. We observed that a left invertible matrix is right invertible iff it is square. We showed that if A is square, then Gaussian elimination applied to [A|I] yields [I|A^-1]. Quiz 5.

We first discussed the arithmetic of partitioned matrices. (Everything works as expected, with the obvious restrictions.) We focused briefly on block diagonal matrices and block upper triangular matrices, noting that a matrix of either type is invertible iff its diagonal blocks are invertible.

Second, we discussed the fact that Gaussian elimination applied to a matrix A produces a factorization A=LR where L is a square invertible matrix and R is in echelon form. If Gaussian elimination can be performed without row switches or row scaling, then L is lower triangular with 1's on the diagonal. If A is square, then R is upper triangular (in which case we call it U instead of R). Thus, if Gaussian elimination applied to a square matrix A without row switches or row scalings succeeds, then it produces a factorization A=LU with
(i) L lower triangular with 1's on the diagonal, and
(ii) U upper triangular.

We observed that if A=LU, then the single system Ax=b is equivalent to a pair of simpler systems Ly=b and Ux=y. We mentioned that if A is n×n, then it takes O(n³) arithmetic operations to solve Ax=b for a given b via Gaussian elimination. It takes O(n³) arithmetic operations to factor A as LU. But it takes only O(n²) arithmetic operations to solve each of Ly=b and Ux=y. Thus, if you want to solve many systems of the form Ax=b for the same A but different b's, then it can be more efficient to factor A first.

We introduced the Leontief economic model, x=Cx+d, which depicts relationships between different sectors of an economy.

We discussed affine transformations, and how to represent them in homogeneous coordinates.

We discussed subspaces of ℝⁿ, Col(A), Row(A), Nul(A), ordered bases for a space, standard basis for ℝⁿ, dimension, rank and nullity of a matrix. We gave algorithms for finding bases for Col(A), Row(A), Nul(A).

We proved the rank + nullity theorem and that dimension is well defined. We worked on some practice problems.

We discussed properties of the determinant including: the determinant of (block) triangular matrices, determinant of a product is the product of determinants, the determinant can be computed via Gaussian elimination, if T(x)=Ax, then the determinant of A measures the "volume expansion" associated with T, and that the determinant is the unique alternating multilinear function d of n variables defined on ℝⁿ for which d(e₁,…,e_n)=1. We briefly discussed Cramer's Rule.

We discussed the word "abstract", then defined (real) vector spaces. We discussed the new examples M_m×n(ℝ), P_n(t), and C^k([0,1]). We computed that M_m×n(ℝ) has dimension mn, P_n(t) has dimension n+1, and C⁰([0,1]) is infinite dimensional.

We discussed coordinates relative to a basis, and found coordinates for certain vectors in M_2×2(ℝ) and P₃(t). We defined the (very important!) terms isomorphism and isomorphic for real vector spaces. We proved the (very important!) theorem that any finitely generated real vector space is isomorphic to ℝⁿ for some n.

We reviewed the proof that any finitely generated real vector space is isomorphic to ℝⁿ for some n. We then explained why, if V and W are finitely generated vector spaces with ordered bases B and C respectively and T:V→W is a linear tranformation, then there is a unique matrix _C[T]_B such that

_C[T]_B [v]_B=[T(v)]_C.

We calculated this matrix for some examples, namely the matrix for the transpose operation from M_2×2(ℝ) to itself and the matrix for the derivative operation from P_n(t) to P_n-1(t). We mentioned that if I:V→V is the identity transformation, then _C[I]_B is the change of basis matrix from basis B to basis C.

We defined eigenvectors, eigenvalues and eigenspaces for linear transformations and for matrices. Our intuition is that eigenvectors point in "preserved directions", and that eigenvalues measure the amount of "stretching" in a preserved direction. We explained why λ is an e-value of A iff A-λI is noninvertible, and why V_λ = Nul(A-λI). We showed that the e-values of a triangular matrix are exactly the diagonal values. Quiz 9.

We defined the characteristic polynomial χ_A(λ) of a matrix A. We noted that if the entries of A belong to a number system ℤ, ℚ, ℝ, ℂ (or any other commutative ring), then the coefficients of χ_A(λ) belong to the same number system.

We discussed the structure of the roots of a polynomial p(λ) of degree n over ℂ versus ℝ. In particular, a complex polynomial of degree n always has n complex roots counting multiplicity, but a real polynomial of degree n may have AT MOST n real roots (possibly fewer than n). Consequently, a complex or real n×n-matrix has exactly n complex eigenvalues counting multiplicity, but a real n×n-matrix has AT MOST n real eigenvalues (possibly fewer).

We proved that a transformation T:V→V is diagonalizable iff V has a basis of e-vectors for T, and that the corresponding e-values appear on the diagonal, in the proper order, in the diagonal form for T. We worked through some examples of diagonalizable and nondiagonalizable transformations.

We defined the complex number system. We described the planar representation of complex numbers and defined the real and imaginary parts of a complex number, and the "conjugate", "absolute value", and "argument" of a complex number. We noted that if A is a real matrix, then its nonreal e-values come in conjugate pairs. If λ is a nonreal e-value, then the conjugation map is a real vector space isomorphism from the e-space V_λ to its conjugate V _λ. This holds for generalized e-spaces too. Thus we can deal with complex e-values and e-spaces in conjugate pairs when we eliminate the use of complex numbers (which shall be described Friday).

Do practice problems:
Section 5.5: 24, 25
Chap 5 Supp.: 1, 2, 5, 7, 15, 17

We described how to put real matrices in approximate diagonal form (or approximate JCF) while avoiding the use of complex numbers.

We started discussing how to compute length and angle in ℝⁿ. We defined the dot product and showed that for vectors in the plane the length of vector u is ||u||:= (u•u)^1/2, and the angle θ between vectors u and v satisfies u•v=||u|| ||v||cos(θ). We used this formula to show that the angle between two adjacent diagonals on the face of a cube meet at 60 degrees.

Do practice problems:
Section 6.1: 1, 5, 7, 9, 13, 17, 19, 20
Section 6.2: 1, 3, 17, 23, 24, 27, 29
Section 6.5: 3, 17, 18, 22, 25

We reviewed the relationships between dot product, length and angle. We mentioned the link between dot product and matrix multiplication, namely u•v = u^Tv, which allows us to derive the basic arithmetical laws of dot product from the arithmetical laws of matrix multiplication. We discussed how to find a unit vector in a given direction. Then we turned to a discussion of orthogonality. We defined u⊥v ⇔ u•v = 0⇔ u^Tv = 0. We defined the orthogonal complement of a set of vectors, and explained why orthogonal complements are subspaces. We observed that Nul(A) = Row(A)^⊥, and pointed out that this observation yields an algorithm for finding orthogonal complements. (Namely, if S is a set of vectors, make them the rows of a matrix A, use the nullspace algorithm to find S^⊥=Nul(A).) We stated without proof that in a finite dimensional space, every subspace equals its double complement. We defined orthonormal bases, and defined a matrix to be orthogonal if its columns form an orthonormal basis. We mentioned that orthogonal matrices are exactly the matrices of linear transformations that preserve length and angle. (In fact, orthogonal matrices are exactly those that preserve dot product. This means that O is orthogonal iff u•v = (Ou)•(Ov) for all vectors u and v.) A compact definition of orthogonal matrix is: a matrix O satisfying O^TO = I. Examples of orthogonal matrices include rotation matrices and permutation matrices.

The second part of the lecture focused on the problem of finding the best approximate solution to an inconsistent system. These are systems of the form Ax=b where b∉Col(A). A best approximate solution, or least squares solution, is any solution to the normal equations, which are the equations derived from Ax=b by left multiplying by A^T: namely, A^TAx=A^Tb. We explained why any solution to A^TAx=A^Tb is a solution to Ax=b̂ where b̂ is the orthogonal projection of b onto Col(A). Quiz 12.

Do practice problems:
Section 6.3: 5, 11, 21, 22
Section 6.4: 9, 17, 18

We worked out a least squares problem about fitting a curve to a set of data. Then we developed a formula for calculating the orthogonal projection of a vector onto a 1-dimensional subspace. We used the formula to show how to orthogonalize a basis for a subspace (Gram-Schmidt). Finally we extended our earlier formula to one for calculating the orthogonal projection of a vector onto an arbitrary subspace. I distributed a review sheet for the final.