Singular Value Decomposition (SVD) – Complete Guide
Singular Value Decomposition (SVD) is one of the most fundamental matrix factorizations in linear algebra. It plays a key role in machine learning, data compression, numerical computation, and matrix analysis.
1. Definition and Formula of Singular Value Decomposition
For any real matrix \( A \) of size \( m \times n \), its SVD is a factorization:
\[ A = U \Sigma V^{T} \]
Where:
- \( U \) is an \( m \times m \) orthogonal matrix
- \( V \) is an \( n \times n \) orthogonal matrix
- \( \Sigma \) is an \( m \times n \) diagonal matrix with non-negative diagonal entries
The diagonal entries of \( \Sigma \) are called the singular values of \( A \): \[ \sigma_1 \ge \sigma_2 \ge \cdots \ge 0 \]
SVD always exists for every real matrix, even if the matrix is not square or is singular.
2. Examples
Example 1
Compute the SVD of the matrix:
\[ A = \begin{pmatrix} 3 & 0 \\ 0 & 4 \end{pmatrix} \]Show Answer
This matrix is already diagonal. Therefore:
\[ U = \begin{pmatrix} 1 & 0 \\ 0 & 1 \end{pmatrix}, \quad V = \begin{pmatrix} 1 & 0 \\ 0 & 1 \end{pmatrix} \] \[ \Sigma = \begin{pmatrix} 3 & 0 \\ 0 & 4 \end{pmatrix} \]This is the SVD.
Example 2
Compute the SVD of:
\[ A = \begin{pmatrix} 0 & 2 \\ 2 & 0 \end{pmatrix} \]Show Answer
\[ A = U\Sigma V^T \] \[ U = \frac{1}{\sqrt{2}} \begin{pmatrix} 1 & 1 \\ 1 & -1 \end{pmatrix}, \quad V = U \] \[ \Sigma = \begin{pmatrix} 2 & 0 \\ 0 & 2 \end{pmatrix} \]All singular values equal 2.
3. Common Mistakes and Tips
- Do not confuse SVD with eigenvalue decomposition — SVD works for all matrices, not only square ones.
- Singular values are always non-negative.
- \( U \) and \( V \) are orthogonal, meaning their columns are orthonormal vectors.
- The diagonal matrix \( \Sigma \) may not be square.
4. Practice Problems (with Answers)
Exercise 1. Find the singular values of \[ A = \begin{pmatrix} 5 & 0 \\ 0 & 1 \end{pmatrix} \]
Show Answer
\[ \sigma_1 = 5, \quad \sigma_2 = 1 \]Exercise 2. Compute the SVD of \[ A = \begin{pmatrix} 1 & 0 \\ 0 & -3 \end{pmatrix} \]
Show Answer
Singular values: \[ \sigma_1 = 3,\quad \sigma_2 = 1 \]Exercise 3. Determine the rank of the following matrix using Singular Value Decomposition (SVD):
\[ A = \begin{pmatrix} 4 & 2 & 1 \\ 2 & 1 & 0 \\ 1 & 0 & 0 \end{pmatrix} \]Show Answer
The singular values of \(A\) are approximately: \[ \sigma \approx (5.000,\; 0.999,\; 0.000). \] There are two non-zero singular values, so: \[ \text{rank}(A) = 2. \]
Exercise 4. Use SVD to determine the rank of the matrix:
\[ B = \begin{pmatrix} 3 & 3 & 6 \\ 1 & 1 & 2 \\ 2 & 2 & 4 \end{pmatrix} \]Show Answer
The singular values of \(B\) are approximately: \[ \sigma \approx (8.246,\; 0.000,\; 0.000). \] Only one singular value is non-zero, so: \[ \text{rank}(B) = 1. \]
Singular value decomposition is often studied together with the matrix inverse, matrix multiplication, and pseudo inverse.
Copyright Notice: This article is original content from the Matrix Calculator website. Please credit the source when sharing or reproducing it. For more matrix computation tools, visit matrixcalcu.com.
