prolem in sparse

hi,

to avoid out of memory I used sparce matrix

x=sparce(10,10);
x(1:10,1:10)=5;

when I need to store this matrix in file

how do that

where the x be as:

   (1,10)       5
   (2,10)       5
   (3,10)       5
   (4,10)       5
   (5,10)       5

when I create file to store this matrix ,was not created

thanks

[Sven]

"when I create file to store this matrix ,was not created"

Did you create it or not? If you *tried* to create it, what did you try? And what went wrong? Please show exactly what you tried.

[huda nawaf]

I created sparse matrix but don't now how store it in txtfile

Answer 1

[Sven]

Huda, try this (edited to update with comments):

% Make Data
x=sparse(10,10);
x(1:10,1:10)=5;

% Get the rows, cols, values of any non-zero elements
[rows,cols,vals] = find(x);

% Save to a text file
fid = fopen('myFile.txt','w')
fprintf(fid, '(%d,%d)\t%f\n',[rows,cols,vals]')
fclose(fid)

And then to later open and read it:

fid = fopen('myFile.txt','r');
[R,C,V] = textscan(fid, '(%d,%d)\t%f');
fclose(fid)

Keep in mind a few things:

1. If you only want to save these files then read them back into MATLAB, it's much simpler to use save() and load(). For example:

myData = {[2 5 32], [1 2 3 4 5 6 7], [12 23 34]};
save('myFile.mat', 'myData')
clear myData       % This just deletes the variable "myData"
load('myFile.mat') % This just loads it again!

2. If you're setting the value of every element in a sparse matrix to a non-zero value, then you shouldn't be using a sparse matrix in the first place... sparse matrices are the most efficient when the majority of elements in your matrix are zeros. Instead, you should use a cell array of vectors like this:

myData = {[2 5 32], [1 2 3 4 5 6 7], [12 23 34]};
for i = 1:length(myData)
  for j = 1:length(myData{i})
     fprintf('Cell #%d, Element #%d, equals %f\n', i, j, myData{i}(j))
  end
end

[huda nawaf]

thanks,

i did, now if need to return array from file

what i have to do?

[Sven]

So you atually just want to save from MATLAB to a file and then load it into MATLAB later?

Have you tried simply using the save() and load() functions?

x=sparse(10,10);

x(1:10,1:10)=5;

save('myData.mat','x')

clear all

load('myData.mat')

[huda nawaf]

hi,

before I did what you suggest now.

I will came back for the comment before the last one

when I save the sparse in txtfile, I did not get the all rows.I got just the second row, and the first was ignored .Why?

arr1=sparse(1000,L*2);

for i=1:2 %%%%%%% just try for two loops

arr=[c1 dat.'];%%%%c1 dat vectors ,i want place them in matrix,in each loop the two vectors will change.

arr1(i,1:L*2)=reshape(arr.',1,[]);

end

[m,n]=size(arr1);

[rows,cols] = find(arr1);

rowsColsVals = [rows cols full(arr1(find(arr1)))];

% Save to a text file

fid = fopen('arr1.txt','w')

for k=1:n

for k1=1:m

fprintf(fid, '(%d,%d)\t%d\n',rowsColsVals');

end

fprintf(fid,'\n');

end

fclose(fid)

[Walter Roberson]

It is inefficient to keep assigning values to a sparse matrix like that. It is more efficient to build a list of all of the values and provide the list of indices and values in the sparse() call.

The code Sven gave is inefficient. Instead of the code

[rows,cols] = find(x);

rowsColsVals = [rows cols full(x(find(x)))];

use

[rows,cols,values] = find(x);

rowsColsVals = [rows,cols,values];

[huda nawaf]

thanks,

i got it , but i store just values in text files. i don't need rows and cols as long as it is ordered sequentially:

(1,1) 1488844

(1,2) 732561

(1,3) 822109

Right?

[Walter Roberson]

If it is ordered sequentially, you probably should not be using sparse().

[Sven]

@Walter: Hah, cool... I'd never used the 3rd argument from find before. I realised that my double-call to find() was one more than necessary...

[huda nawaf]

I have 174000 records, I segmented it into three parts,and by using sparse I read these data and again place it into three files .

Now , I have three files store the indices and values.

The problem now is when I read each file alone and place it in array again, then integrate all arrays in one array (for processing I need all my data be in one array), I'm facing out of memory problem again before integration. i.e when read each file alone and place it in array ,I get this message of "out of memory"

What I have to do

fid = fopen('d:\matlab7\work\flixsterdata\finalflix0.txt','r');

c= textscan(fid, '(%d,%d)\t%f');

fclose(fid);

>> c1=c{1};c2=c{2};c3=c{3};

>> for i=1:length(c1)

x(c1(i),c2(i))=c3(i);

end

??? Out of memory. Type HELP MEMORY for your options.

>>

thanks

[Walter Roberson]

Don't do that. Just

x = sparse(c1, c2, c3);

[huda nawaf]

I got this:

??? Function 'sparse' is not defined for values of class 'int32'

i looked for solution for this problem before send this comment,

i try to convert from class to class but i did not find "typecast function" in ver. 7

[Walter Roberson]

x = sparse(double(c1),double(c2),c3);

Or better yet, change your textscan format to use %f instead of %d

[huda nawaf]

i don't know why my previous query regarding this problem is lost

i told u that segmented my files into three parts ,i read it using sparse matrix and there is no problem. but when try to group them in one array, again get out of problem.

thanks

[Walter Roberson]

"Show older comments" to see what you wrote before.

You did say that you got "out of memory", but in that case you were creating your sparse matrix first and then assigning in to it a bit at a time. That is inefficient (slow) and requires more memory than if sparse() is given all of the information at one time.

Question: are your elements still stored sequentially like you mentioned above,

(1,1) 1488844

(1,2) 732561

(1,3) 822109

If so then using sparse arrays uses more memory than using regular arrays. sparse arrays are only useful when a fair number of the matrix entries are 0 .

[huda nawaf]

thank walter,

look please,

Now I have three files , the structure of each file as:

(1,1) 57699

(2,1) 18858

(3,1) 5806

In command window, I tried to read this file and convert it to array.

I wrote in command win.

c=textscan(fid,'(%d,%d)\t%d\n');

c1=c{1};c2=c{2};c3=c{3}

for i=1:length (c)

a(c1(i),c2(i))=c3(i);

end

again get out of memory

Fangjun suggest in indices and values thread

to use

[I,J,V]=textread('test.txt','(%f,%f) %f');

A=zeros(2,3);

A(sub2ind(size(A),I,J))=V

but when preallocation A as A(1000,200000), I will get

??? Error using ==> zeros

Out of memory. Type HELP MEMORY for your options.

[huda nawaf]

Walter, do u mean if my array did not contains zero's ,it is not necessary to use sparse matrix.

In fact I have no zeros in my data, but because the rows are not equal in their lengths , so the zeroes are inserted to be the rows are equal.

in other words, you said I have not to create sparse at first may slow the running, and said I have to use

sparse(c1,c2,c3).

what I want to define another array as sparse instead of c

do u mean , when write arr=sparse(1000,200000)

is not right?

what I have to do?

[Sven]

Huda, if you have no zeros in your actual data, then I can guarantee that you do *not* want to use sparse().

If I understand correctly, I think your data is best stored in a cell array of vectors. Something like:

data = {[2 5 32], [1 2 3 4 5 6 7], [12 23 34]};

Finally, can you please tell us _why_ you want to save your data to a text file? If you need to open it in another program, maybe .txt is the way to go. If you need to open it in MATLAB again later, there are *much* better options than .txt.

[huda nawaf]

i want to say , before ask i look for

please,regarding cell

if i want to do the following with cell, how do it?i don't know how cell works

for i=1:5

for j=1:not defined

a(i,j)=any value

end

end

regarding txt file

no, i don't need open it in another program

i use it in matlab only.

what other options,and why?

do u mean creat .m or .dat in stead of .txt , if so why? what i will get in this case

[Sven]

Huda, I've updated the answer again to include how to use a cell. Please read the doc section on "Access Data in a Cell Array".

You should use load() and save() because it is the *simplest* way to save and load your data. You don't need to worry about converting to text or converting from text... all this is handled by MATLAB.