Help with using parfor (possible indexing problem)

I've created a program which essentially is a block processing version of the xcorr command enabling you to use the command without getting out of memory errors when using large data.

My data (the input and template file) comes from two mat files on desktop. Each block (of a 1D array) of the input is read in, processed (Applying xcorr command to produce 1D array of length 2M-1), and the output (this 2M-1 length array) is saved back into another mat file via block processing

(What makes this trickier is that due to the concept of how xcorr works, to get the correct output, all the the blocks need to overlap by ~50% where all the elements from the second block are added on to the elements they overlap from the first, and so on).

My run time was approximately 15 hours however and I really want to try and shorten this down (ironically, my computer can handle using xcorr without block processing and it takes several minutes to complete).

I understand that for parfor to work, I need each iteration to be independent of each other (because they are not processed in order). I also need to make sure my indexing is simplistic (which is where my problem lies). Below is a slightly simplified version of my code which processes all the blocks apart from the first and last (which are done separately without a loop).

inputData=matfile('DataToInput.mat'); %file to test for correlation
template=matfile('template.mat'); %the template used for correlation

r=0; %initialise the saving
save('output.mat','r','=v7.3') %save new mat file for output of program
m=matfile('output.mat','Writable',true);

x=template.x;
len1=length(inputData.y);
len2=length(template.x);

for N=2:floor(len1/len2)
  y=inputData.y((N-1)*len2+1:N*len2,1);
  r=xcorr(y,x);
  m.r=[m.r; zeros(len2,1)];
  m.r(end-(2*len2-2):end,1)=m.r(end-(2*len2-2):end,1) +r; 
end

if I replace "for" with "parfor" I get the following issues:

I get an orange warning for y=inputData.y((N-1)*len2+1:N*len2,1); stating: the entire array structure 'inputData' is a broadcast variable.This might result in unnecessary communication overhead.

and I get a red warning for m.r=[m.r; zeros(len2,1)]; stating: Valid indices for 'm' are restricted in PARFOR loops

I've tried preallocating m (I know the final length of m) outside the loop (which required changing the indexing of the last line in the loop, because of the "end" bit) but I'll still get a red warning for that last line.

Any help?

Answer 1

[Edric Ellis]

I would suggest making a few changes to your code that should eliminate all the code analyzer warnings / variable classification errors. Firstly, parfor unfortunately isn't always great at understanding indexing expressions into fields of structs, so I would avoid using them for input and output. Secondly, your parfor loop appears to be accessing inputData.y in a conceptually "sliced" manner, but unfortunately your indexing expressions are too complicate for parfor to understand. Your output m.r is a problem because you're updating elements as you go along. You need to convert that to a "reduction" output. Something like this:

num_iter = floor(len1/len2) - 1;
inputData_y = reshape(inputData.y, len2, []);
m_r = zeros(len2 * (1+num_iter), 1);
parfor N = 2:floor(len1/len2)
    y = inputData_y(:, N);
    tmp = zeros(len2 * (1+num_iter),1);
    tmp(((N-2)*len2 + 2):N*len2) = xcorr(y, x);
    m_r = m_r + tmp;
end

[MatlabUser]

Thank you. Apologies if I did not mention this, but the reason I was block processing xcorr instead of just using just using the command once was because I was using big data which could crash computers if i did it in one go.

I was wondering whether there are other possible ways to use parfor other than the way you mentioned?

In my code, the significant variables that take up the most matlab memory would be y,and r, both of which are just a fraction of the size of inputData.y (I actually clear these variables at the end of the loop but neglected to include them here for simplicity)

In your second line of parfor, not only have you rcreated tmp variable will be as large as the inputData.y but you're creating that variable in every iteration of the loop. I don't know how fast parfor is compared to a conventional for loop, but I'd guess any time saving would be lost.

For some perspective, inputData.y is a 1x37million double array. your tmp variable will be creating this in every iteration.

[MatlabUser]

Instead of having one long 1D array for m_r. I thought it might be better to just store the result of xcorr in different columns (slices) of m_r, similar to how you reshaped inputData.y.

I've applied the code you wrote above the parfor. Knowing the dimensions of m_r, I changed the r=0 (when creating the m_r matfile) to r=zeros(2*len-1,floor(len1/len2))

in the parfor:

y=inputData.y(:,N);
 r=xcorr(y,x);
 m.r(:,N)=m.r(:,n) + r;

For some reason I STILL get the orange broadcast warning for y=inputData.y(:,N); and the red error for m.r(:,N)=m.r(:,n) + r;

The indexing has been simplified so why is this still happening?

[Edric Ellis]

As I said in my original answer, unfortunately parfor isn't great at understanding indexing expressions into struct types. So, you should try using simple variables instead of structs for inputData and m.