Fitting Spaces

In this chapter, we generalize line fitting in 2D and plane fitting in 3D to general dimensions and consider how to fit subspaces and affine spaces to a given set of points in nD. A subspace is a space spanned by vectors starting from the origin, and an “affine space” is a translation of a subspace to a general position. The fitting is done hierarchically: we first fit a lower dimensional space, starting from a 0D space (= a point), then determine a space with an additional dimension so that the discrepancy is minimized, and continue this. This principle corresponds to what is known as “Karhunen-Loeve expansion” in signal processing and pattern recognition and as “principal component analysis” in statistics. The fitted space is computed from the spectral decomposition of a matrix, which we call the “covariance matrix,” but also can be obtained from its singular value decomposition. We point out that the use of the singular value decomposition is more efficient with smaller computational complexity.