Subspace Learning
Algorithms are taken from the book Multilinear Subspace Learning: Dimensionality Reduction of Multidimensional Data.
A survey of the field can be found at http://www.dsp.utoronto.ca/~haiping/Publication/SurveyMSL_PR2011.pdf.
code
The multilinear principal component analysis (MPCA) is an extension of the conventional PCA for data with multiple modes. It produces one projection matrix for each data mode, that projects its corresponding mode to a smaller size. Using all projection matrices consecutively results in a tensor with the same mode count, but smaller (or same) mode sizes. There exists no closed form solution for the MPCA, so an alternating partial projection optimization is used. It usually converges after a few iterations. The benefit compared to the PCA is, that the computationally expensive eigendecomposition is not computed for the covariance matrix of all data elements, but for the covariance matrix of each mode separately.
code
One potential drawback of the MPCA is, that the projected features, in contrast to the PCA, may be correlated. As an algorithm that takes advantage of the multiple modes, but nonetheless produces uncorrelated features, the uncorrelated principal component analysis (UMPCA) was introduced. The UMPCA produces a specified count of independent, uncorrelated features. These features are not organized in multiple modes, but as a vector. For each feature, the algorithm creates a elementary multilinear projection (EMP), that projects the data tensor to a scalar. Again, UMPCA uses alternating partial projection, and an additional constraint ensuring independency of the features.
Both MPCA and UMPCA produce the standard PCA when used on data with one mode.
original MNIST digits:
PCA:
MPCA:
UMPCA:
The multilinear PCAs don't make much sense for these tiny two-dimensional MNIST images. They are designed for very high dimensional, multi-modal data.