A "moving window" matrix inversion?

I have a tall, skinny column matrix, H. It has N rows. I want to compute a "moving matrix inverse" along H'*H, using L << N rows at a time. That is, I want to produce another matrix, iH, that looks like this at sample k:

iH = inv( H(k:k+L-1,:)'*H(k:k+L-1,:) );

I know inv.m isn't what I want to use, I just want to communicate the idea, and it's more cumbersome to write A\eye(size(A,2)).

One "dumb" option is to use an anonymous function:

invStat = @(k) inv( H(k:k+L-1,:)'*H(k:k+L-1,:) );

and to do:

invStat = cellfun(invStat, num2cell(1:N-L));

But when N is huge (like 1e5), and L is small (like 1e1) this is ridiculously slow. I have no special toolboxes (other than stats and signal processing).