Kinect streaming Color/Depth sync with IMAQ

Hi,

I'm trying to capture color and depth image at the 30FPS (Kinect default) and to save it on the disk.

The problem is to capture have the synchronization between both device and the same number of frames. Right now, I'm not able to achieve that really good.

Always have less frame or time sync problem or can only do it on a really short acquisition like 2 second. But I would like to do it like for a minute and maybe more.

Anyone have something to point ?

      % code
colorVid = videoinput('kinect',1);
depthVid = videoinput('kinect',2);
colorVid.FramesPerTrigger = inf;
depthVid.FramesPerTrigger = inf;

Code for the disk logger and after I just to something like that :

    % code
start([colorVid depthVid]);
wait([colorVid depthVid], 10);
stop([colorVid depthVid]);

and this is simple but still doesn't work well. I want to change the wait(...,10) with a GUI button start/stop later on, but first I would like to make it work well like that.

On disk, I took .avi for color and .mj2 for depth as specify by the doc.

thanks in advance,

Alex

[Alexandre Bizeau]

Update : I change my logging function -> http://www.mathworks.com/matlabcentral/fileexchange/4792-custom-file-format-logging

So now, I do have both stream in .bin file. It working for like 5-6FPS and for less then 3 minutes. But after that, same thing, I'm getting more frame on Color than the depth sensor and losing synchronization.

And more FPS = less time. For 30FPS around at 20-25 second the frame change appear.

I'm searching for a better memory managing or something 'manual' trigger in a while loop maybe or something. But nothing right now...

Answer 1

[Anchit Dhar]

Hi Alexandre,

In order to achieve synchronization between the two streams, you should use manual triggering. The current synchronization issue that you observe is due to the delay between starting of the two objects.

The following example shows how to use manual triggering with the two streams:

http://www.mathworks.com/help/imaq/acquiring-from-the-color-and-depth-devices-simultaneously.html

In this way, both the objects are start and wait for trigger to begin the logging of frames.

-Anchit

[Alexandre Bizeau]

Yes, I tried this solution and I'm using a part of it right now.

Still setting

colorVid.FramesPerTrigger = inf;
depthVid.FramesPerTrigger = inf;

And When I do start, after I'm using matlab pause(2) to be sure both sensor are start and sync.

Once this pause is over, I do the trigger. Since it's an infinite trigger. It logging data until the stop button is pressed.

But for a speed of 30 FPS and approx. 1 minute of record, the frame start to unsync sometime and at the end you can see it well.

Their about 20-40 frames differences at the end. So like recommenced on my stackoverflow post Here.

I create a small script that trying to match frame by frame depending on time. Just adding a double value on every logging isn't taking time so it's work well. Right now, it's post-processing because it's what I need for 30 FPS. But I'll try to make it work for 10 FPS in live acquisition.

For a ~1 minute video, It ~1800 frames (normally a bit more) and my script drops about 10-25 frames in total. (Depth or color are delete depending if the elapse time between frame is nothing respecting some condition).

So yes, this is a part of the answer, because I need to use it to have a great sync on start, but my script delete some frames at the begin too, it just matching time and dropping some frames.

• Answer accepted because it the beginning of what you need for a good sync.