H.264/MPEG-4 video over network

Hi.

There is good documentation out there on both Image Acquisition Toolbox and Instrument Control Toolbox (that of tmtool), but it lacks documentation on how to combine these. I have a network camera (specifically that on the ar.drone) that streams H.264 video over TCP/IP. Apparently one can combine these two toolboxes so that tmtool can read the video stream and use that object in the Image Acquisition Toolbox. I would be very greatfull if anyone could shed some light on this matter. If any other toolbox could help me, my university have most toolboxes, so that should not be an issue.

Thank you.

[sky nku]

Do you know how to connect to a network camera using Instrument Control Toolbox? Thanks!! For example, the ip addresss of my camera is 192.168.3.253，and my PC connects to the camera with a simple net cable( UTP=UNSHIELDED TWISTED PAIR )

Answer 1

[Walter Roberson]

I have not found any documentation that suggests that the Image Acquisition Toolbox is able to handle streamed H.264. All of the information about H.264 that I have found so far is that files that contain H.264 can be processed (by VideoReader in R2011b or later, or by the Computer Vision Toolbox in R2012b or later).

The only known work-around at the moment is to stream the H.264 to a different (non-MATLAB) program that decodes it and can pass it on in a format that MATLAB can handle already. A small number of people have reported being able to implement this, but information about which intermediate program they used or which image format they ended up using is in short supply.

James was chasing this but did not find the mechanism by early 2012; see http://www.mathworks.com/matlabcentral/answers/26324

[Christoffer Stausland]

Hi, and thank you for the swift reply. I thought that since it supports H.264 files it would support streaming as well. There is an API with C examples doing this that I can use, but as always it would be just so very much simpler just using Matlab.

Answer 2

[Mark Jones]

Hi,

If you know C++ well, then I think your best approach would be to find a library that supports reading and decoding the stream (possibly GStreamer) and then developing an adaptor for the Image Acquisition Toolbox.

http://www.mathworks.com/help/imaq/creating-custom-adaptors.html

Mark

[Walter Roberson]

Darn, I have been hoping that I was just reading the documentation too strictly, or had overlooked something. Several people have asked for this facility.

[Christoffer Stausland]

I agree with you there, but I only use C++ for scientific purposes, so video decoding with libraries is something I have to read up on. My geekness in me is telling me to spend time on this, 'cause if I get the video stream into Matlab the possibilities using a quadrocopter (in this case) are so many. Using image recognition in Matlab is indeed very simple and relativly fast.

Thank you both for the input, I will check out custon adaptors as well.

Answer 3

[Florian Enner]

I know this thread is very old, but it does show up quite high when searching for h264 streaming. In case someone else comes across this problem, I've uploaded a submission that supports streaming h264 (among other formats) from ip cameras.

% Connect to stream
cam = HebiCam('<address>');

% Continously display latest image
figure();
fig = imshow(getsnapshot(cam));
while true
    set(fig, 'CData', getsnapshot(cam)); 
    drawnow;
end

You can get it from FileExchange or Github