How to use image segmentation in video images of folders

Hello.

I have several videos. After convert videos to binary images, I want to convert binary images to centered images(cropped).

Binary images ---> cropped image

1.I will use this code to convert video to binary images.

clear all
close all
%// read the video:
reader = VideoReader('person01_boxing_d1_uncomp.avi');
vid = {};
while hasFrame(reader)
    vid{end+1} = im2single(readFrame(reader));
end
%// simple background estimation using mean:
bg = mean( cat(4, vid{:}), 4);
%// estimate foreground as deviation from estimated background:
for i=1:40
    fIdx(i) = i; %// do it for frame 1 ~ 60
    fg{i} = sum( abs( vid{fIdx(i)} - bg ), 3 );
    fg{i} = imresize(fg{i}, 0.2);
    fg{i} = reshape(fg{i},[],1);
end
X = cell2mat(fg);
data = double(X);

2. And then I want to convert binary images to centered image(cropped) using image segmentation

https://www.mathworks.com/matlabcentral/fileexchange/25157-image-segmentation-tutorial

Please let me know how to use these codes when I use several videos as input.

I attached sample video.

message = sprintf('Would you like to crop out each coin to individual images?');
reply = questdlg(message, 'Extract Individual Images?', 'Yes', 'No', 'Yes');
% Note: reply will = '' for Upper right X, 'Yes' for Yes, and 'No' for No.
if strcmpi(reply, 'Yes')
	figure;	% Create a new figure window.
	% Maximize the figure window.
	set(gcf, 'Units','Normalized','OuterPosition',[0 0 1 1]);
	for k = 1 : numberOfBlobs           % Loop through all blobs.
		% Find the bounding box of each blob.
		thisBlobsBoundingBox = blobMeasurements(k).BoundingBox;  % Get list of pixels in current blob.
		% Extract out this coin into it's own image.
		subImage = imcrop(originalImage, thisBlobsBoundingBox);
		% Determine if it's a dime (small) or a nickel (large coin).
		if blobMeasurements(k).Area > 2200
			coinType = 'nickel';
		else
			coinType = 'dime';
		end
		% Display the image with informative caption.
		subplot(3, 4, k);
		imshow(subImage);
		caption = sprintf('Coin #%d is a %s.\nDiameter = %.1f pixels\nArea = %d pixels', ...
			k, coinType, blobECD(k), blobMeasurements(k).Area);
		title(caption, 'FontSize', textFontSize);
	end

Answer 1

[Image Analyst]

See the FAQ.

Put your code for one video file inside the loop over all video files. It might be best to put it all inside a function that takes the video filename.

[Kong]

Thank you so much.

I will read videos using this code.

Could you let me know how I can use your image segmentation tutorial with this code?

%// read the video:
list = dir('*.avi')
% loop through the filenames in the list
for k = 1:length(list)
    reader = VideoReader(list(k).name);
    vid = {};
    while hasFrame(reader)
        vid{end+1} = im2single(readFrame(reader));
    end
    %// simple background estimation using mean:
    bg = mean( cat(4, vid{:}), 4);
    %// estimate foreground as deviation from estimated background:
    for i=1:100
        fIdx(i) = i; %// do it for frame 1 ~ 60
        fg{i} = sum( abs( vid{fIdx(i)} - bg ), 3 );
        fg{i} = imresize(fg{i}, 0.2);
        fg{i} = reshape(fg{i},[],1);

    X = cell2mat(fg);
    data = double(X);
end

[Image Analyst]

Somewhere in there you'll need to get a mask of foreground objects - a binary image. When you have that call regionprops() and then just crop out all the blobs, either displaying them or saving them with a unique filename, whatever you prefer.

for k1 = 1:length(list)
    % Get number of frames into maxFrames
    for frameNumber = 1 : maxframes
        % For each frame number.....
    end
end

Inside your loop over frameNumber, do your segmentation to get your binary image (mask) for that frame, and then ...

blobMeasurements = regionprops(mask, 'BoundingBox', 'Area');
numberOfBlobs = length(blobMeasurements)
% Make sub folder for all the blobs of all the frames for this video file.
thisFileName = list{frameNumber};
[f, baseFileNameNoExt, ext] = fileparts(thisFileName);
outputFolder = fullfile(f, [baseFileNameNoExt, '/outputFolder']);
if ~isfolder(outputFolder)
    mkdir(outputFolder)
end
% Let user know where the cropped files will go.
message = sprintf('Results for file %s\nwill go into folder\n%s\n', baseFileNameNoExt, outputFolder)
uiwait(helpdlg(message))
plotRows = ceil(sqrt(numberOfBlobs));
% Loop over all blobs in this frame displaying them and writing them to disk.
for k = 1 : numberOfBlobs           % Loop through all blobs.
    % Find the bounding box of each blob.
    thisBlobsBoundingBox = blobMeasurements(k).BoundingBox;  % Get list of pixels in current blob.
    % Extract out this coin into it's own image.
    subImage = imcrop(originalImage, thisBlobsBoundingBox);
    % Display the image with informative caption.
    subplot(numberOfBlobs, numberOfBlobs, k);
    imshow(subImage);
    caption = sprintf('Blob #%d  has area = %d pixels', ...
    k, blobMeasurements(k).Area);
    title(caption, 'FontSize', 15);
    % Save to disk in the output folder
    thisCroppedFileName = fullfile(outputFolder, sprintf('Frame %d, Blob %d.png', frameNumber, k));
    imwrite(subImage, thisCroppedFileName);
end

[Kong]

I appreciate the guideline.

Could you test the attached video using this code?

Frankly, I didn't understand your comment. I have lots of errors.

[Kong]

Do I need to save binary images in the sub-folder?

[Image Analyst]

No you don't, unless you want to. I'm not sure why you want the binary images in the first place. I'd think you could just process each binary image as you get it (doing whatever you need to do with them), without saving them to a drive.

[Kong]

Yes. you are right. I don't need to save binary images.

In this your code, what is the "mask"?

blobMeasurements = regionprops(mask, 'BoundingBox', 'Area');

[Kong]

Thank you so much. I got error. Could you explain what list is?

clear all
close all
%// read the video:
vidObj = VideoReader('daria_run.avi');
maxframes = 0;
iwant = cell([],1) ; 
while hasFrame(vidObj)
    F = readFrame(vidObj);    
    maxframes = maxframes + 1;   
    imagesc(F)
    drawnow
    iwant{maxframes} = F ;
end

for k1 = 1:length(list)
    % Get number of frames into maxFrames
    for frameNumber = 1 : maxframes
        % For each frame number.....
        blobMeasurements = regionprops(mask, 'BoundingBox', 'Area');
        numberOfBlobs = length(blobMeasurements)
        % Make sub folder for all the blobs of all the frames for this video file.
        thisFileName = list{frameNumber};
        [f, baseFileNameNoExt, ext] = fileparts(thisFileName);
        outputFolder = fullfile(f, [baseFileNameNoExt, '/outputFolder']);
        if ~isfolder(outputFolder)
            mkdir(outputFolder)
        end
        % Let user know where the cropped files will go.
        message = sprintf('Results for file %s\nwill go into folder\n%s\n', baseFileNameNoExt, outputFolder)
        uiwait(helpdlg(message))
        plotRows = ceil(sqrt(numberOfBlobs));
        % Loop over all blobs in this frame displaying them and writing them to disk.
        for k = 1 : numberOfBlobs           % Loop through all blobs.
            % Find the bounding box of each blob.
            thisBlobsBoundingBox = blobMeasurements(k).BoundingBox;  % Get list of pixels in current blob.
            % Extract out this coin into it's own image.
            subImage = imcrop(originalImage, thisBlobsBoundingBox);
            % Display the image with informative caption.
            subplot(numberOfBlobs, numberOfBlobs, k);
            imshow(subImage);
            caption = sprintf('Blob #%d  has area = %d pixels', ...
            k, blobMeasurements(k).Area);
            title(caption, 'FontSize', 15);
            % Save to disk in the output folder
            thisCroppedFileName = fullfile(outputFolder, sprintf('Frame %d, Blob %d.png', frameNumber, k));
            imwrite(subImage, thisCroppedFileName);
        end
    end
end

[Image Analyst]

That was your variable. Remember when you had

%// read the video:
list = dir('*.avi')

but now you decided not to use it anymore. It's also a built-in function in a toolbox that you don't have so it's warning you about that. I suggest you use another name like fileList instead.

[Kong]

Thank you.

I want to approach step-by-step.

First of all, I tried to get binary images. fg{i} is each binary image (1 ~ 28 image)

When I use the below code, I got this error.

Could you check the code below?

clear all
close all
%// read the video:
list = dir('*.avi')
% loop through the filenames in the list
for k = 1:length(list)
    % Get number of frames into maxFrames
    reader = VideoReader(list(k).name);
    vid = {};
    while hasFrame(reader)
        vid{end+1} = im2single(readFrame(reader));
    end
    %// simple background estimation using mean:
    bg = mean( cat(4, vid{:}), 4);
    %// estimate foreground as deviation from estimated background:
    for i=1:28
        fIdx(i) = i; %// do it for frame 1 ~ 60
        fg{i} = sum( abs( vid{fIdx(i)} - bg ), 3 );
        fg{i} = imresize(fg{i}, 0.2);
        % For each frame number.....
        blobMeasurements = regionprops(fg{i}, 'BoundingBox', 'Area');
        numberOfBlobs = length(blobMeasurements)
        % Make sub folder for all the blobs of all the frames for this video file.
        thisFileName = list{i};
        [f, baseFileNameNoExt, ext] = fileparts(thisFileName);
        outputFolder = fullfile(f, [baseFileNameNoExt, '/outputFolder']);
        if ~isfolder(outputFolder)
            mkdir(outputFolder)
        end
    end
end