Error: "SVD does not support sparse matrics. Use SVDS to compute a subset of the singular values and vectors of a sparse matrix." while training SOM.

Hi

I want to train SOM using text features (TFIDF). I am using Neural Net clustering (nctool) app in MATLAB 2019a for this purpose. My data is of size 2866 x 61695 (2866 instances and each containing 61695 elements) and also sparse. When I tried to train the SOM using this data then it gave me an error "SVD does not support sparse matrics. Use SVDS to compute a subset of the singular values and vectors of a sparse matrix." I did not understand this. Please let me know how can I train my SOM with this data.

Thanks in advance.

Answer 1

[Bjorn Gustavsson]

It means that your SOM-training function somewhere uses (or tries to use) the svd function. That function expects a full matrix, if it is called with a sparse matrix throws the error you got. If you feed your training-data in sparse format you might get around that by calling the SOM-training function with full(your_data) and hope for the best - this might work or the SVD-call might run out of memmory.

HTH

[Parminder Kaur]

Thank you for the reply. I understood your first two lines, however, unable to exactly understand the last line. I am using my whole data as training data and feeding it into SOM for training but the issue here seems to be of sparse data acceptance. I need a work around (if there is any) for training SOM with my huge sparse text feature data using nctool available in MATLAB. If it cannot be done using this tool then how it can be done?

[Bjorn Gustavsson]

My suggestion was that you should convert your data from sparse to full with the hope that that might work (since your data is rather bit you might run out of memmory), to do that you would modify your function-call from using your 2866 x 61695 data in the sparse variable your_data (what you've named your variables I cannot know) to full(your_data).

If that doesn't work you'll have to reduce the size of your input-data somehow (train on sub-sets, reduce dimensionality of data, something more clever...)

[Parminder Kaur]

Thank you so much. It worked.