About Singular Value Decomposition
A matrix of size m × n is a grid of real numbers consisting of m rows and n columns.
In linear algebra, a branch of mathematics, matrices of size m × n describe linear mappings from n-dimensional to m-dimensional space.
The word linear roughly means that straight lines map to straight lines and the origin in n‑dimensional space maps to the origin in m‑dimensional space.
When we have an (m × n)‑matrix A and a (n × k)‑matrix B, we can compute the product AB which is an (m × k)‑matrix.
The mapping corresponding to AB is exactly the composition of the mappings corresponding to A and B respectively.
Singular Value Decomposition (SVD) states that every (m × n)‑matrix A can be written as a product
where U and V are orthogonal matrices and the the matrix Σ consists of descending non-negative values on its diagonal and zeros elsewhere.
The entries σ1 ≥ σ2 ≥ σ3 ≥ … ≥ 0 on the diagonal of Σ are called the singular values (SVs) of A.
Geometrically, Σ maps the j‑th unit coordinate vector of n‑dimensional space to the j‑th coordinate vector of m‑dimensional space, scaled by the factor σj.
Orthogonality of U and V means that they correspond to rotations (possibly followed by a reflection) of m‑dimensional and n‑dimensional space respectively.
Therefore only Σ changes the length of vectors.
