How to save and preview live frames from videoinput at the same time? (GUI + GigE camera)

Hello Everybody!

I'm working on an GUI application to monitor a 3D Printer. The application must show the live 3D Printing process while each frame of the GigE Camera is saved as an image in a given directory. These monitoring sessions can go from 1 minute to almost 120hours! (It is also possible to configure a delay in the monitoring session, supponsing some minor maintenance has to be done to the 3D Printer before it actually starts printing). I'm using a Mako G-131 GigE camera and the base structure for my code is the example ' myCameraGUI.m ' provided by the MathWorks Support Team.

At the moment, inside my OpeningFcn I have declared the videoinput with an immediate trigger. Also the preview function enables the user to see the live images from the video input.

    function myCameraGUI_OpeningFcn(hObject, eventdata, handles, varargin)
    % This function has no output args, see OutputFcn.
    % hObject    handle to figure
    % eventdata  reserved - to be defined in a future version of MATLAB
    % handles    structure with handles and user data (see GUIDATA)
    % varargin   command line arguments to mycameragui (see VARARGIN)

    % Choose default command line output for mycameragui
    handles.output = hObject;
    handles.video = videoinput('gentl', 1,'Mono8');

    set(handles.video,'TimerPeriod', 0.05, ...
    'TimerFcn',['if(~isempty(gco)),'...
    'handles=guidata(gcf);'... % Update handles
    'image(getsnapshot(handles.video));'... % Get picture using GETSNAPSHOT and put it into axes using IMAGE
    'set(handles.cameraAxes,''ytick'',[],''xtick'',[]),'... % Remove tickmarks and labels that are inserted when using IMAGE
    'else '...
    'delete(imaqfind);'... % Clean up - delete any image acquisition objects
    'end']);

    %immediate trigger - ocurrs automatically immediately after the start function is issued
    triggerconfig(handles.video,'immediate');

    %Show video on app startup
        axes( handles.cameraAxes );
        vidRes = handles.video.VideoResolution;
        nBands = handles.video.NumberOfBands;
        hImage = image( zeros(vidRes(2), vidRes(1), nBands) );
        %setappdata(hImage,'UpdatePreviewWindowFcn',@mypreview_fcn);
        preview(handles.video, hImage);

    % Set the default FPS to the maximum - to prevent errors from previous runs
        camera_info = getselectedsource(handles.video);
        camera_info.AcquisitionFrameRateAbs = camera_info.AcquisitionFrameRateLimit;

    % Show information about the current acquisition to the user
    set(handles.edit7,'String',camera_info.DeviceTemperature);
    set(handles.edit9,'String',camera_info.AcquisitionFrameRateAbs);

    %Colour the start acquisition button in green
    set(handles.startAcquisition,'BackgroundColor','g');

    % Update handles structure
    guidata(hObject, handles);

    % UIWAIT makes mycameragui wait for user response (see UIRESUME)
    uiwait(handles.myCameraGUI);

Up to this point everything goes as planned. However, as soon as I click the startAcquisition button two possible scenarios occur (Depending if I'm using function 'drawnow' or 'imgshow(data)'). If I use the 'drawnow' function after having saved the images with 'imwrite' function, then the cameraAxes will update constantly (as desired), but the saved images will only appear on the folder location after the entire acquisition is done.

On the other side, if I use the 'imgshow(data)' after having saved the images with the 'imwrite' function, then the cameraAxes will stop updating ...but the saved images will appear as they are being created in the folder location.

% --- Executes on button press in startAcquisition.
function startAcquisition_Callback(hObject, eventdata, handles)
% hObject    handle to startAcquisition (see GCBO)
% eventdata  reserved - to be defined in a future version of MATLAB
% handles    structure with handles and user data (see GUIDATA)

%Recover object video
running_camera = getselectedsource(handles.video);
axes(handles.cameraAxes);

%recover global variables from text fields - from user
global hd md hr mr fps na videodata ruta

resultInputData = verifyInputFormat(hObject, eventdata, handles);

%This function checks input format from text fields 
if (resultInputData~=0)

      % Start/Stop acquisition button is pressed
      if strcmp(get(handles.startAcquisition,'String'),'Iniciar Adquisicion')

          %Small feedback to user to let him know the program is running
          set(handles.startAcquisition,'BackgroundColor','b');
          set(handles.startAcquisition,'String','En Espera');
          pause(1);

          %Wait until the delay configured by the user is elapsed
          hr_enSegundos = hr * 60 * 60;
          mr_enSegundos = mr *60;
          tiempoDeRetraso = hr_enSegundos + mr_enSegundos;
          pause(tiempoDeRetraso);

          %Once the delay is finished change the color of the acquisition button
          set(handles.startAcquisition,'BackgroundColor','r');
          set(handles.startAcquisition,'String','Detener Adquisicion');
          set(handles.edit1,'Enable','off');
          set(handles.edit2,'Enable','off');
          set(handles.edit3,'Enable','off');
          set(handles.edit4,'Enable','off');
          set(handles.edit5,'Enable','off');
          set(handles.edit6,'Enable','off');
          set(handles.edit8,'Enable','off');
          set(handles.saveRoute,'Enable','off');

          %Recover the duration for the monitoring session
          hd_enSegundos = hd * 60 * 60;
          md_enSegundos = md *60;
          tiempoDeGrabacion = hd_enSegundos + md_enSegundos;
          %Convert duration to seconds and to frames
          framesDeGrabacion = tiempoDeGrabacion * fps;

          %Capture all frames within 1 second
          handles.video.FramesPerTrigger = fps;
          %Set fps based on user input
          running_camera.AcquisitionFrameRateAbs = fps;
          %Trigger will execute as many (seconds -1) as the duration set by the user
          handles.video.TriggerRepeat = framesDeGrabacion-1;

          %Update user information on GUI 
          set(handles.edit9,'String',running_camera.AcquisitionFrameRateAbs);
          set(handles.edit7,'String',running_camera.DeviceTemperature);

          %Start camera
          start(handles.video)
          i = 1;

          while(isrunning(handles.video))

              while(handles.video.FramesAvailable >= 1)
              data = getdata(handles.video,1);
              numDeImagen = num2str(i);
              nomImagen = strcat(ruta,'\',na,numDeImagen,'.jpg');
              nomImagenString = num2str(nomImagen);
              imwrite(data(:,:,:,1) ,nomImagenString);
              i=i+1;

              *%Update cameraAxes
              imshow(data);
              %drawnow        %not working!*

                  if i > framesDeGrabacion

                      stop(handles.video);

                  end 

              end 

          end 

          %Natural end of acquisition
          saveAndFinishRoutine(hObject, eventdata, handles);

      else

          %If acquisition is stopped by user 
          saveAndFinishRoutine(hObject, eventdata, handles);

      end

end

So, if anybody could please tell me a better approach to save and preview frames at the same time.. it would be awesome.

The following two images are printscreens from my application moments after clicking button StartAcquisition. As you can see, the live image Axes FREEZES with the first frame. However, the data acquired in the saved images is succesfully updated based on the camera recording.

After starting the acquisition:

Little before the 60 seconds have passed.

Any help will be appreciated.

[Geoff Hayes]

Bego - rather than using drawnow, try using pause instead. Perhaps something like

 imwrite(data(:,:,:,1) ,nomImagenString);
 i=i+1;
 % Update cameraAxes
 imshow(data);
 pause(0.01);

[Bego]

Thanks for your answer Geoff!

I made the changes you suggested and made some tests with acquisitions lasting for 1 minute. For the cases in which the FPS was set to a value smaller than 8FPS your solution worked fine!

However, for the cases in which the FPS had values greater than 9FPS, the acquisition started to 'lag' and the acquisition lasted more than it should (additional frames were logged as well).

So, for what I have seen it seems the problem is somewhere else. Some sort of de-synchronisation between what I see and what is logged. I'll have to read the documentation and keep looking.

Many thanks,

[Geoff Hayes]

Bego - what is the FPS, frames per second? It could be that the pause of 0.01 seconds (10 milliseconds) is too long and so needs to be shortened.

[Bego]

Hi Geoff! - I did tried with several values for the pause, from the tiniest 0.00001 up to 0.1 ...and none of them appeared to give me the correct synchronisation.

[piku mandal]

how to run this ?

Answer 1

[Bego]

Although good ideas were provided by Geoff, what I did to solve my issue was a change of approach. After much reading I learned that the start() function isn't as asynchronous as it is thought ..instead this function is similar to a while() loop since it takes ownership of the execution if a proper trigger is configured. So, what I did was to re-write my startAcquisition_Callback() so that:

 handles.video.FramesPerTrigger = fps;
 %fps is a variable I get from the user's interface 
 %this variable means frames per second

 handles.video.TriggerRepeat = monitoringTimeInSeconds-1;
 %monitoringTimeInSeconds is a variable I get from the user
 %this variable is the sum of hours and minutes converted to seconds

 handles.video.FramesAcquiredFcnCount = 1;
 %This specifies that one by one frames will be acquired

 set(handles.video,'FramesAcquiredFcn',@display_frame);
 %This specifies that function display_frame will be the
 % one in charge of managing incoming frames.

 start(handles.video);

Then, within my display_frame function I take care of frames one by one:

%Retrieve frames one by one
imageData = getdata(obj,1);
%Draw each frame
drawnow;
%Save frames to images
imageNumber = num2str(imgCount);
imwrite(imageData, strcat(saveRoute, '\Photo',imageNumber,'.jpg'));
imgCount = imgCount +1;

This approach has the benefit that once all frames specified are captured (once condition handles.video.TriggerRepeat = monitoringTimeInSeconds-1 is met) the video object will automatically stop (similar to a stop(handles.video).

Yup! Thanks to those changes I'm now able to configure a monitoring session with variable FPS in which the live display is updated constantly and the image files appear with almost no delay on the desired folder location.

Right on! (*high fives myself in the air) :)

[Andy Scott García Gordillo]

Hello @Bego, I have been struggling with this same problem for some time now. I am trying to write a graphical interface to control a microscope camera as well as other devices together. Like a week ago I found this thread and I realized this will solve all my problems; I haven't found anything alike in the web. Please, would you send me the entire code for me to understand better how you solve your issue? It would be also of great help to use it as a template to code my GUI. You can contact me here: [e-mail]. Thank you. -Andy García.