circshift slower on GPU

Unlike other matrix operations, it seems that circshift runs slower on GPU than on CPU.

Run the following code, GPU spends twice the time of CPU. Any method to improve it?

M = rand(512);
N = gpuArray(rand(512));

tic;
for i = 1:10
    circshift(M, [10, 10]);
end
toc;

tic;
for i = 1:10
    circshift(N, [10, 10]);
end
toc;

Answer 1

[Joss Knight]

Yes, circshift is a bit slow on the GPU particularly when it has to shift both rowwise and columnwise, because that means it has to stride the entire array reading and writing out to new locations - the GPU isn't especially good at that in comparison to main memory. You can see the GPU performance overtake the CPU somewhere around arrays of size 1000x1000. On my Kepler card:

>> A = rand(1000); B = gpuArray(A);
>> timeit(@()circshift(A,[10 10]))

ans =

      0.0018

>> gputimeit(@()circshift(B,[10 10]))

ans =

      0.0020

>> A = rand(5000); B = gpuArray(A);
>> timeit(@()circshift(A,[10 10]))

ans =

      0.0266

>> gputimeit(@()circshift(B,[10 10]))

ans =

      0.0053

I'm not sure if I can advise how to improve it without writing your own version. Usually if you want to shift the data in a simple way you can reimplement it as some indexing and concatenation operations, which may well turn out to be faster for your particular use case.

Nevertheless I'll treat this as a request to improve the performance of circshift on the GPU.

[Jiang]

Thank you Joss. I wish we will see an improved circshift soon. I realized that in most cases I don't need shifting the matrix circularly. I just need to shift the matrix along one direction and pad the other size with zero. So I wrote this simple function, it will shift the 2D matrix unidirectionally. And it turns out to run much faster.

function N = gpuShift(M, k)
% shift 2D array on GPU
N = zeros(size(M), 'single', 'gpuArray');
if k(1) >= 0  && k(2) >= 0
    N(k(1)+1:end, k(2)+1:end) = M(1:end-k(1), 1:end-k(2));
elseif k(1) >= 0  && k(2) < 0
    N(k(1)+1:end, 1:end+k(2)) = M(1:end-k(1), 1-k(2):end);
elseif k(1) < 0  && k(2) < 0
    N(1:end+k(1), 1:end+k(2)) = M(1-k(1):end, 1-k(2):end);
elseif k(1) < 0  && k(2) >= 0
    N(1:end+k(1), k(2)+1:end) = M(1-k(1):end, 1:end-k(2));
else
    error('wrong shift values');
end
end

[Joss Knight]

If you only need to shift off the end then use shiftdim (to shift left) or reshape (to shift right).