Efficient summing of parts of an array

4 ビュー (過去 30 日間)
Ife
Ife 2023 年 7 月 24 日
コメント済み: Ife 2023 年 7 月 25 日
I have a 4D dataset that I'm expressing as a sparse 2D array of 1s and 0s. The 4D data has dimensions NPixHeight by NPixWidth by NBins by NFrames, and as a sparse array has the dimensions NPixHeight*NFrames by NPixWidth*NBins.
As part of my pre processing of I want to compress the 2D array by summing along the F dimension in groups, the size of the group is defined by nExps in the below code. To realise this I create and transform matrix T that after multiplication with the original array outputs a NPixHeight*nExps by NPixWidth*NBins array.
This last step is pretty slow for our planned application and I was hoping someone might be able to point me in the right direction to speed it up.
Here's an excerpt from the code
%% Data Reshape
% number of exposures for macro time
nExps = NFrames/macroFrames;
% generates sparse array of counts Ypix*NFrames by Xpix*Bins where
% NFrames is limited by macrotime
yNew = y+(f-1).*NPixHeight;
xNew = x+(b-1).*NPixWidth;
SpCounts = sparse(yNew,xNew,1,NPixHeight*NFrames,NPixWidth*NBins); % SLOW
% transform matrix that sums a number frames set by macro time
T = repmat(speye(NPixHeight,NPixHeight),1,macroFrames);
TCell = repmat({T}, 1, nExps);
clearvars T
T = blkdiag(TCell{:});
% multiplication and reshaping to output
% 4D array of Y x X x Bins x NumberMacroExposures
camData.counts = sparse(size(SpCounts,2),size(T,1));
camData.counts = full(T*SpCounts).'; % SLOW
camData.counts = reshape(camData.counts,NPixWidth*NBins,NPixHeight,nExps);
camData.counts = reshape(pagetranspose(camData.counts),NPixHeight,NPixWidth,NBins,nExps);
(I know the double reshape is awkard but its not enough of a bottleneck to bother fixing)
  3 件のコメント
Ife
Ife 2023 年 7 月 24 日
編集済み: Ife 2023 年 7 月 24 日
It depends on the user input but for a "worst case" with:
NPixHeight = 192
NPixWidth = 126
NBins = 344
NFrames = 247360
macroFrames = 30920
nExps = 8
gives a sparse array of 47493120 x 43344 with a sparsity of 6.2715e-05.
Forgive the stupid question but what is a mex routine...?
the cyclist
the cyclist 2023 年 7 月 24 日
編集済み: the cyclist 2023 年 7 月 24 日
A MEX file is a function that calls C,C++, or Fortran from within MATLAB.

サインインしてコメントする。

採用された回答

Matt J
Matt J 2023 年 7 月 24 日
I would recomend that you maintain the 4D array in ndSparse form instead,
at which point you can do,
temp = reshape(yourArray,[],NFrames)*kron(speye(nExps),ones(macroFrames,1));
result=reshape(temp,NPixHeight, NPixWidth , NBins, []);
  1 件のコメント
Ife
Ife 2023 年 7 月 25 日
Thanks! Due to what I'm using this code for I can skip the 4D sparse representation then reshape steps and readout my data directly into the correct size/orientation. My code is running 2x - 5x faster depending on the size of the datasets now!

サインインしてコメントする。

その他の回答 (0 件)

カテゴリ

Help Center および File ExchangeLogical についてさらに検索

製品


リリース

R2022a

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by