pca as an optimization problem

For some reasons, I would like to compute PCA (or, rather the eigenvectors and eigenvalues of a (de-meaned) sample covariance matrix ) using an oprimization function. I think I understood how to set up the objective functions and constraints, but struggling to actually implement it with Matlab. I am referring the first answer: What is the objective function of PCA? .

Could anyone help me to understand how to set up the problem with Matlab?

[John D'Errico]

Can I suggest this is a really bad idea? Why do you think you want/need to do this? If your goal is just to learn about PCA then a far better idea is to learn the necessary linear algebra, and thus to understand why any brute force optimization is not even needed at all. The linear algebra is not really that sophisticated, and compared to what you will need to get into trying to do this as an optimization, you will be far ahead.

Just pick up any basic book on PCA and read the first few chapters. Jackson's book is my preference, but then Ted and I worked together when I was just starting my career.

[Image Analyst]

Why not just use the built-in pca() function, if you have the Statistics and Machine Learning Toolbox? See attached demos.

[yp78]

Thank you for both your advice. I understand the related linear algebra, but I am looking at the optimization side as I need to seek some extension of PCA for some other applications, such as imposing a different type of constraint on the objective function etc. (it may no longer be a PCA then).

Or say, I just would like to know the setting up of an optimization problem with MATLAB, and PCA would be a good starting point for my particular needs.

Answer 1

[Prabakaran G]

As per my understanding, PCA can used to reduce the number of input predictors, which make the problem is simpler and generalize. Also, it reduce the complexity of the function to optimize.