Factor Analysis and extensions
We consider this data model and several extensions where the noise covariance matrix has a more general structure, such as banded, sparse, block-diagonal, and cases where we have multiple data covariance matrices that share the same noise covariance matrix. Starting from a nonlinear weighted least squares formulation, we propose new estimation algorithms for both classical FA and its extensions. The optimization is based on Gauss-Newton gradient descent. Generally, this leads to an iteration involving the inversion of a very large matrix. Using the structure of the problem, we show how this can be reduced to the inversion of a matrix with dimension equal to the number of unknown noise covariance parameters.
This results in new algorithms that have faster numerical convergence and lower complexity compared to several maximum-likelihood based algorithms that could be considered state-of-the-art. The new algorithms scale well to large dimensions and can replace eigenvalue decompositions in many applications even if the noise can be assumed to be white.
The attached matlab code was used to generate the figures in the paper. It also contains some alternative algorithms that were used in its comparison