running time

hi,

the running time of the following code is very long:

%%%%%%%%%%%%%%
for i=1:p
    count=0;
    for j=1:p
        if mat(i,2)~=0
        if mat(i,2)==mat(j,2)
            mat(i,2)=0;
        mat2(i,2)=i;
        count=count+1;
        x(i)=count;
        end
        end
    end
end
%%%%%%%%%%%%%

where p=211000

are there anyway to make it faster?

thanks in advance

Answer 1

[Walter Roberson]

You overwrite x(i) with count each time the condition matches within "j", but you do not set x(i) at all if the condition never matches, so unless it is important that x not be any longer than the last match found, you can move the assignment down to below the end of the "j" for loop and do it unconditionally.

There may be other changes that will be easier to see once you have done this rewrite.

[huda nawaf]

hi walter,

I reduced my code , then you can detect the problem.Also, I placed comments

f=fopen('ws.txt','r+');%%%%%% open the original data file

z=fscanf(f,'%d');

fclose(f)

mat = reshape(z(:),4,[]).' ;

mat2=reshape(z(:),4,[]).';

L=length(z)/4;

mat2(:,2)=0; mat2(:,3)=[];

mat(:,3)=[];mat3(:,3)=[];

mat(:,3)=[];mat3(:,3)=[];

%%%%%%%%%%%%%%%%%%%%%%%%grouplens movie

[p,o]=size(mat);id=1;

for i=1:p %%% try to look for any value in mat matrix and their

%%%%frequency , then give it unique id

if mat(i,2)~=0 %%%% to avoid looking for zero values

k=1;

for j=1:p

x=sprintf('%d',mat(i,2));

x1=sprintf('%d', mat(j,2));

if mat(j,2)~=0 %%%% to avoid looking for zero values

if x==x1

if i~=j

mat(j,2)=0; %%%%%% any value have been searched will be ignored

mat2(j,2)=id; %%% give it id

end

end

end

id=id+1;

end

end

f1=fopen('y1.txt','w');

for i=1:p

for j=1:o

fprintf(f1,'%d ',mat2(i,j));

end

fprintf(f1,'\n');

end

fclose(f1)

I'm facing problem because I badly these values to be stored in file.

thanks

Answer 2

[Daniel Shub]

Since mat(i, 2) will always equal mat(j, 2) when i is equal to j and you will always set mat(i, 2) to zero unless it is already equal to zero. I think at the end you have effectively done

 mat(:, 2) = 0;

the mat2(i, 2) part is basically

 mat2(:, 2) = 1:p;
 mat2(mat(:, 2) == 0, 2) == 0;

This leaves the count part. You are looking for how many matches of mat(i,2) there are in mat(:, 2). I think you can do this with a hist function.

n = hist(mat(:, 2), unique(mat(:, 2));

The final step is putting the n's into x. I think there are probably quick ways of doing this.

[huda nawaf]

thanks, I have added some changes on my code where it is necessary for my goal , and I did what walter say.

But, the problem is long running time ,and out of memory .

the problem is my data is very large(data of social net.).

are there solution for this problem?

Daniel, i think what you suggest earlier is no longer worh with my new code.

please I need solution.

f=fopen('ws.txt','r+');%%%%%% open the original data file

z=fscanf(f,'%d');

fclose(f)

mat = reshape(z(:),4,[]).' ;

mat2=reshape(z(:),4,[]).';

L=length(z)/4;

mat2(:,2)=0; mat2(:,3)=[];

mat(:,3)=[];mat3(:,3)=[];

mat(:,3)=[];mat3(:,3)=[];%mat2(:,2)=0;

%%%%%%%%%%%%%%%%%%%%%%%%grouplens movie

[p,o]=size(mat);id=1;

for i=1:10

if mat(i,2)~=0

count=0;

k=1;

for j=1:p

x=sprintf('%d',mat(i,2));

x1=sprintf('%d', mat(j,2));

if mat(j,2)~=0

if x==x1

if i~=j

mat(j,2)=0;

end

mat2(j,2)=id;

count=count+1;

ind(i,k)=j;

k=k+1;

end

end

end

x2(i)=count;

id=id+1;

end

end

f1=fopen('y1.txt','w');

for i=1:p

for j=1:o

fprintf(f1,'%d ',mat2(i,j));

end

fprintf(f1,'\n');

end

fclose(f1)

[Daniel Shub]

The inability to format code in a comment means I will not answer you here. I apologize since this isn't your fault. If Walter has answered your original question, then accept his question and ask a new one with your new code. If he hasn't, but has helped to refine your question, edit your question where you can add code markup. On a side note, trim your code as much as possible. I am not going to be able to get past the first line since I do not have ws.txt (nor do I want it).