The Eckart-Young Theorem

The Eckart-Young Theorem: Given an n by m matrix X of rank r £ m £ n, and its singular value decomposition, ULV', where U is an n by m matrix, L is an m by m diagonal matrix of singular values, and V is an m by m matrix such that

U'U = In and V'V = VV' = Im

with the singular values arranged in decreasing sequence

l1³l2³l3³ ... ³lm³ 0

then there exists an n by m matrix B of rank s, s £ r, which minimizes the sum of the squared error between the elements of X and the corresponding elements of B when

B = ULsV'

where the diagonal elements of the m by m diagonal matrix Ls are

l1³l2³l3³ ... ³ls > ls+1 = ls+2 = ... = lm = 0

Geometrically, the Eckart-Young Theorem states that the least squares approximation in s dimensions of a matrix X can be found by replacing the smallest m-s roots of L with zeroes and remultiplying ULV'.

For example, given:



the rank 2 approximation is:




svd_example.r -- R Program to illustrate simple SVD.
svd_matrix.txt -- Simple 4 by 3 matrix.
svd_example.r looks like this:
#
# file: svd_example.r
#
# Purpose: Simple R program showing Singular
#          Value Decomposition
#
#
rm(list=ls(all=TRUE))
#
#
rc.file