Fast reshaping or squeezing

I'm dealing with big matrix, such as dimension X,Y and Z are respectively around 300, 6 and 200000.

The point is that I'm doing backward regressions along the first dimension starting from row 300 up to the first. In my opinion I can't factorize more. So a for loop is needed for the backward regression.

I'm then using each submatrix A of size 1x6x200000 to perform operations,such as - but not only - A*A' so I need to reshape A to dimension 6x200000.

And here's the problem, this operation made 300 times takes a lot of time overall. Is there any fast solution that could improve the computation time ? Using squeeze gives the same, FYI.

Answer 1

[Guillaume]

I need to reshape A to dimension 6x200000.

I assume you mean you need to reshape the 1x6x200000 submatrix to 6x200000.

reshaping should always be near instantaneous. All it involves is changing the header of the matrix to store the new size of each dimension. The actual content of the matrix stays untouched and does not need shuffling around.

What would take time however is the slicing, the extracting of the 1x6x200000 matrix out of the 300xXxY full matrix since that involves going over the whole matrix and picking out every 300th value. I would think you may be able to get a small gain of performance if you swapped the order of the dimensions:

newbigmatrix = permute(bigmatrix, [2 3 1]);  %results in a 6x200000x300 matrix. Will take a while!
for page = size(newbigmatrix, 3):-1:1
   submatrix = newbigmatrix(:, :, page);  %extract a 6x200000 slice
   %...
end

The advantage of that new order is that the elements of each submatrix are already contiguous so the slicing should be faster. It also cuts out the reshape but as said, that shouldn't be what took time in the first place.

[cedric W]

The initial computation time is around 5s

Permuting takes 0.75s and then slicing total time is around 0.9s

So it divided by 3 which is quite a good improvement, nice one !

As a rule of thumb, should I always set the last dimension as the one to be sliced ?

[Matt J]

You can also possibly avoid extracting slices by donwloading MTIMESX and multiplying as a stack

AAtranspose=mtimesx(newbigmatrix, newbigmatrix,'t');