Advice on Speeding up Loops

I need to run the code below with the parameter amnt equal to about 10^8. The loops are very expensive and I have written in a way to time the execution as a function of the parameter, amnt. The figure shows the execution time as a function of the matrix sizes (amnt) and was run with 12 helpers (cores). Via regression it seems when amnt=10^8 it will take over a month to run.

In a nutshell, the purpose of this code is to bin 4 matrices and then find the most common bin for each.

I am asking:

1. Can anyone recommend a way to vectorize the loop(s) or another way to speed it up?
2. The use of LocB in the ismember function is being flagged by Matlab as being inefficient when using a parfor loop. I don't understand how to make better use of it, i.e. fix it. Any suggestions?

Thanks!

% Test problem for troubleshooting

rang = 100;
sz = 10^8; 

% Some random matrices for this test case.
mat1t = randi(rang,[sz 1]) + 50*rand([sz 1]);
mat2t = randi(rang,[sz 1]) + 40*rand([sz 1]);
mat3t = randi(rang,[sz 1]) + 30*rand([sz 1]);
mat4t = randi(rang,[sz 1]) + 20*rand([sz 1]);

times = [0,0,0,0,0,0,0,0];

for tm1 = 1:length(times) % Loop just used for timing purposes.

amnt = 5^tm1; % Ultimately I want this to be 10^8, i.e. the entire matrices.   

%amnt = numel(mat1(:)); % ALL
mat1 = mat1t(1:amnt); mat2 = mat2t(1:amnt); mat3 = mat3t(1:amnt); mat4 = mat4t(1:amnt);

NP = 50;
dl = max([range(mat1),range(mat2),range(mat3),range(mat4)])/NP;
tol = dl/2;
low = min([min(mat1),min(mat2),min(mat3),min(mat4)]);
high = max([max(mat1),max(mat2),max(mat3),max(mat4)]);
gridp = [low:dl:high]';

datap = [mat1 mat2 mat3 mat4];
whatBin = zeros(size(datap));

tic
for k = 1:4 % Loop over the 4 matrices
    
    gridp1 = [gridp zeros(size(gridp))];
    datap1 = [datap(:,k) zeros(size(datap(:,k)))];
    tolp1 = [tol Inf];
    
    [LIA,LocB] = ismembertol(gridp1,datap1,1,'ByRows',true,'OutputAllIndices',true,'DataScale', tolp1);
    
    parfor kj = 1:length(mat1(:)) % Loop over all matrix elements
        for kkj = 1:numel(LIA) % Loop over all bins
            if ismember(kj,LocB{kkj}) % The use of LocB here is flagged by Matlab as being inefficient. How to fix?
                whatBin(kj,k) = kkj;
                break
            end
        end
    end
end % End of loop over all 4 matrices

% Find the unique rows of the whatBin matrix
[u,I,J] = unique(whatBin, 'rows', 'first');
npix = size(u,1);
paretoPix = zeros([npix 1]); % Number of pixel occurances for a vector of metrics.
 parfor j = 1:npix
     paretoPix(j) = nnz(all(whatBin == u(j,:),2));
 end
 Results = [u paretoPix]; Results = sortrows(Results,-1*(size(whatBin,2)+1));

times(tm1) = toc;

end % end of times loop

semilogx(5.^(1:length(times)),times,'-o')
xlabel('Matrix Elements')
ylabel('Time (s)')

In this figure, the matrix size is the parameter amnt. Using these values in the curve fit app, the regression equation (2nd order polynomial) indicates that when amnt=10^8 it will take ~month!

[Matt J]

In a nutshell, the purpose of this code is to bin 4 matrices and then find the most common bin for each.

If so, why not just use histcounts?

In any case, it's not really clear from the code what the binning operation is and what determines the bin geometry. A visual illustration might make it easier t advise.

[Paul Safier]

@Matt J I agree, without some explanation, the code's goal is confusing. I sketched a fictitious example in the attached image. mat1, mat2 and so on can be thought of as attributes of many (10^8) people. Each row of matrix datap would correspond to a person. The attributes, e.g. height, weight, etc are made into bins. For simplicity, the code I posted just uses one array of bin edges, called gridp. In reality I'd use different ones for each attribute. The goal is to find the most common (most amount of people) set of attributes. See the bar chart.

histcounts is certainly quick, but it doesn't tell me which people are in which bins.

This code works to do what I want, but my thinking is that it can be more efficient--as is, it is too slow to solve for 10^8 rows (people) in a reasonable amount of time.

[Rik]

I'm on mobile, so your image is not much help.

Can't you use the third output of histcounts? That would tell you which persons are in which bin.

[Paul Safier]

@Rik and @Matt J That really made a difference. Thanks for the suggestion!

The full matrix will get done in a reasonable amount of time (hours) now. Thanks.