Eigenvalues and Eigenvectors -- The Fingerprint of a Matrix

When a matrix acts on most vectors, it changes both their direction and length.

But there are always some vectors in "special directions"whose direction remains unchanged after the matrix acts, only the length changesThese are eigenvectors, and the scaling factor is the eigenvalue.

Core formula: \( A\mathbf{v} = \lambda \mathbf{v} \)

\( \mathbf{v} \): eigenvector (non-zero), whose direction remains unchanged after transformation
\( \lambda \): eigenvalue, representing the factor by which v is stretched

λ > 0

Stretching/shrinking in the same direction

Direction remains unchanged

λ < 0

Stretching/shrinking in the opposite direction

Direction is completely reversed

An n×n matrix has at most n linearly independent eigenvectors.

The eigenvalues of a symmetric matrix are always real — this is one reason why symmetric matrices are so important.

Everyday Examples

Stretching a Rubber Membrane

A rubber membrane is stretched by a factor of 2 horizontally, while the vertical direction remains unchanged.

An arrow in the horizontal direction (e.g., (1,0)) is stretched to (2,0), but its direction remains unchanged — it is an eigenvector, λ=2.

An arrow in the vertical direction (e.g., (0,1)) is completely unchanged — it is also an eigenvector, λ=1.

A diagonal arrow (e.g., (1,1)) changes direction — it is not an eigenvector.

Mathematical Definition

\[ A\mathbf{v} = \lambda \mathbf{v} \quad \Leftrightarrow \quad (A - \lambda I)\mathbf{v} = 0 \]

Condition for the existence of a nonzero solution: \( \det(A - \lambda I) = 0 \) (characteristic equation).

Python Hands-on Practice

Example

import numpy as np

A = np.array([[2, 0], [0, 1]]) # Stretch 2x horizontally, unchanged vertically
eigenvalues, eigenvectors = np.linalg.eig(A)

print("Eigenvalues:", eigenvalues) # [2. 1.]
print("Eigenvectors (columns):\n", eigenvectors)

# Verify Av = λv
for i in range(2):
v = eigenvectors[:, i]
lam = eigenvalues[i]
print(f"\nVerify eigenvector{i+1}:)
print(f" A @ v = {A @ v}")
print(f" λ * v = {lam * v}")
print(fEqual: {np.allclose(A @ v, lam * v)})

# Eigenvalues of a symmetric matrix are all real
S = np.array([[1, 4], [4, 2]])
print(f"\nEigenvalues of symmetric matrix S: {np.linalg.eigvals(S)} (all real))

特征值: [2. 1.]
特征向量 (列):
 [[1. 0.]
 [0. 1.]]
验证特征向量1: A@v=[2. 0.], λv=[2. 0.], 相等: True
对称矩阵 S 的特征值: [ 5. -2.]（全实数）

Interactive Eigenvector Demo

Drag the blue vector v and observe the transformation Av (red dashed line) by the matrix A = [[2,0],[0,1]].

When v is horizontal or vertical, Av is collinear with v—these are the eigenvector directions:

Application Scenarios in AI

PCA Dimensionality Reduction

Compute the eigenvalues and eigenvectors of the data covariance matrix. The eigenvectors corresponding to the largest k eigenvalues are the k directions (principal components) with the largest data variance. Project the data onto these directions and discard the low-variance directions to achieve dimensionality reduction. Reducing 784-dimensional MNIST images to 50 dimensions still preserves about 90% of the information.

Spectral Normalization

In GAN training, the discriminator's weight matrix is spectrally normalized at each step: W ← W / \sigma_{max}(W), where \sigma_{max} is the largest singular value (for a square matrix, this equals the absolute value of the largest eigenvalue). This ensures the discriminator is 1-Lipschitz, stabilizing training and reducing mode collapse.

Spectral Methods for Graph Neural Networks

The eigenvalues and eigenvectors of the graph Laplacian matrix L = D - A (D is the degree diagonal matrix, A is the adjacency matrix) define the Fourier transform on graphs. GCN (Graph Convolutional Network) performs convolution in this spectral domain—although modern GCNs mostly use spatial-domain methods, the spectral method is the theoretical foundation.

Other Extensions

Advanced

Application Examples