Linear Algebra Basics You Actually Need
Matrices are just grids of numbers arranged in rows and columns. Vectors are lists of numbers with direction and magnitude. That's the entire vocabulary at the start. Everything else builds from those two ideas. I picked up Introduction To Linear Algebra Johnson because someone on a forum recommended it over Strang for self-study. It covers the same core material—vector spaces, eigenvalues, matrix decompositions—but with slightly less hand-waving around the proofs. The downside is that the exercises are numbered poorly and the answers in the back skip steps you actually need to see. I ended up filling in the gaps with worked solutions from other sources. The book starts with systems of linear equations. You solve them using row reduction, also called Gaussian elimination. This is where most people get tripped up because they memorize the steps without understanding what each operation means geometrically. Each row operation is really just moving the intersection point of planes around until the solution becomes obvious.
Here is a practical edge case I ran into while working through the chapter on matrix inverses. The Johnson text gives an example where a 3x3 matrix looks invertible at first glance. You compute the determinant and it comes out to zero. But the trick is that one of the rows is nearly a scalar multiple of another row—like 0.999 times. Floating point arithmetic can make that matrix appear invertible when it is not. The workaround I used was to compute the condition number before attempting inversion. If it exceeds 1e12, the matrix is numerically singular regardless of what the symbolic determinant says. This saved me from getting garbage results in a signal processing project I was working on.
Vector Spaces and Subspaces
A vector space is a collection of vectors that stay closed under addition and scalar multiplication. That means if you take any two vectors from the space and add them, the result is still in the space. Same thing if you multiply a vector by any real number. Simple to state. Harder to internalize because the definition applies to objects that aren't arrows in space—polynomials, functions, sequences. The four fundamental subspaces of a matrix are the column space, null space, row space, and left null space. Johnson covers these after the determinant chapter, which I thought was backwards. Understanding rank and nullity first would make the subspace discussion much clearer. The rank-nullity theorem states that the dimension of the column space plus the dimension of the null space equals the number of columns. That is the single most useful fact you will carry through every linear algebra course. When you are doing calculations by hand, the fastest way to find a basis for the column space is to row reduce the matrix and pick the original columns that correspond to pivot positions. Do not pick the pivot columns from the reduced matrix. They are different vectors. Pick the originals.
eigenvalues and eigenvectors
An eigenvector of a matrix A is a nonzero vector v such that Av equals lambda times v for some scalar lambda. That scalar is the eigenvalue. The geometric meaning is that the matrix stretches or compresses that particular direction without rotating it. Everything else in the space gets mixed around, but those special directions stay put. Computing eigenvalues by solving the characteristic polynomial det(A minus lambda I) equals zero works fine for 2x2 and small 3x3 matrices. For anything larger you should not attempt this by hand. The characteristic polynomial of a 5x5 matrix has five roots and expanding the determinant manually is a reliable way to make arithmetic errors. Numerical algorithms like the QR iteration are what actual software uses. Johnson includes a section on diagonalization that assumes the reader already knows why it matters. Diagonalization matters because A to the nth power becomes simple when A is diagonalizable. You just raise each eigenvalue to the nth power. This is the engine behind Markov chains, population models, and discrete dynamical systems. Without that context the section feels abstract and pointless.
Orthogonality and Projections
The dot product gives you a way to measure angles and distances in n-dimensional space. Two vectors are orthogonal when their dot product is zero. This is the generalization of perpendicularity to any dimension. Projection onto a subspace uses orthogonality to find the closest vector in that subspace to a given point outside it. The projection formula is p equals A times the inverse of A transpose A times A transpose times b, where the columns of A span the subspace. This requires A transpose A to be invertible, which means the columns must be linearly independent. If they are not, you need the pseudoinverse instead. Johnson mentions the pseudoinverse briefly but does not connect it to the projection formula clearly. I had to look up the derivation separately. Singular value decomposition breaks any matrix into three simpler matrices: U times Sigma times V transpose. This is the most powerful tool in applied linear algebra. It handles rank-deficient matrices, provides the best low-rank approximation, and is the basis for principal component analysis. Most textbooks introduce SVD near the end with a proof that nobody reads. Johnson is no exception, but the application sections that follow are worth working through carefully.
Practical Tips for Getting Through the Material
Do not skip the proof sketches. The intuition behind why Gaussian elimination works is what makes the rest of the book click. Once you understand that row reduction is just systematic elimination of variables, eigenvalues stop feeling like magic formulas. Use a computational tool like NumPy or Octave alongside the book. Running through the examples in code reinforces the connection between the symbolic manipulation and the actual numerical behavior. I spent about three hours converting the first two chapters into Python scripts and that investment paid off when I started seeing patterns in the later material. The exercise difficulty in Johnson jumps around unpredictably. Chapter 4 has several problems that require techniques not yet introduced. When you encounter this, go to the next section, learn the missing tool, then come back. Skipping the hard problems entirely will leave gaps that show up in later chapters.
Get the Full Details
