How to detect Black and white colors in an image?

I'm working on a project that uses a camera to detect black and white blocks in a specific area. Then, it returns the centroid value of each block to be used for a robot arm movement. It works well on python-opencv, but I find many problems using Matlab. In python-opencv I used the Hsv colorspace and set its ranges to easy detect any color and I don't know if it's possible to do the same in Matlab. This code works well with the white color, however it needs high illumination in the area. I tried to decrease the threshold value to detect the black color but it didn't work. So what is the best way to make this detection?

 % Capture the video frames using the videoinput function
 % You have to replace the resolution & your installed adaptor name.
 vid = videoinput('winvideo',1, 'YUY2_1280x720');

 thresh = 0.90;
 % Set the properties of the video object
 set(vid, 'FramesPerTrigger', Inf);
 set(vid, 'ReturnedColorspace', 'rgb')
 vid.FrameGrabInterval = 5;

 %start the video aquisition here
 start(vid)

 % Set a loop that stop after 100 frames of aquisition
 while(vid.FramesAcquired<=100)

    % Get the snapshot of the current frame
    data = getsnapshot(vid);

    bwredFrame = imbinarize(data(:,:,1), thresh); % obtain the white component from red layer
    bwgreenFrame = imbinarize(data(:,:,2), thresh); % obtain the white component from green layer
    bwblueFrame = imbinarize(data(:,:,3), thresh); % obtain the white component from blue layer
    diff_im = bwredFrame & bwgreenFrame & bwblueFrame; % get the common region

    %Use a median filter to filter out noise
    diff_im = medfilt2(diff_im, [3 3]);
    % Convert the resulting grayscale image into a binary image.
    diff_im = im2bw(diff_im,0.18);

    % Remove all those pixels less than 300px
    diff_im = bwareaopen(diff_im,300);

    % Label all the connected components in the image.
    bw = bwlabel(diff_im, 8);

    % Here we do the image blob analysis.
    % We get a set of properties for each labeled region.
    stats = regionprops(bw, 'BoundingBox', 'Centroid');

    % Display the image
    imshow(data)

    hold on

    %This is a loop to bound the red objects in a rectangular box.
    for object = 1:length(stats)
        bb = stats(object).BoundingBox;
        bc = stats(object).Centroid;
        rectangle('Position',bb,'EdgeColor','r','LineWidth',2)
        plot(bc(1),bc(2), '-m+')
        a=text(bc(1)+15,bc(2), strcat('X: ', num2str(round(bc(1))), '    Y: ', num2str(round(bc(2)))));
        set(a, 'FontName', 'Arial', 'FontWeight', 'bold', 'FontSize', 12, 'Color', 'yellow');
    end

    hold off
 end
 % Both the loops end here.

 % Stop the video aquisition.
 stop(vid);

 % Flush all the image data stored in the memory buffer.
 flushdata(vid);

 % Clear all variables
 clear all

[Akira Agata]

You can convert RGB to HSV colorspace by rgb2hsv function. If you could upload your sample image file, I (or someone here) will be able to suggest possible solution to detect the target region.

[TARIK SEV]

Unrecognized function or variable 'hasFrame'.

Error in MotionBasedMultiObjectTracking (line 7)

while hasFrame(obj.reader)

how can we solve this problem

function MotionBasedMultiObjectTracking()

obj = setupSystemObjects()

tracks = initializeTracks();

nextId = 1;

while hasFrame(obj.reader)

frame = readFrame(obj.reader);

[centroids, bboxes, mask] = detectObjects(frame);

predictNewLocationsOfTracks();

[assignments, unassignedTracks, unassignedDetections] = ...

detectionToTrackAssignment();

updateAssignedTracks();

updateUnassignedTracks();

deleteLostTracks();

createNewTracks();

displayTrackingResults();

end

function obj = setupSystemObjects()

obj.reader= videoinput('winvideo',1);

obj.maskPlayer = vision.VideoPlayer('Position', [740, 400, 700, 400]);

obj.videoPlayer = vision.VideoPlayer('Position', [20, 400, 700, 400]);

obj.detector = vision.ForegroundDetector('NumGaussians', 3, ...

'NumTrainingFrames', 40, 'MinimumBackgroundRatio', 0.7);

obj.blobAnalyser = vision.BlobAnalysis('BoundingBoxOutputPort', true, ...

'AreaOutputPort', true, 'CentroidOutputPort', true, ...

'MinimumBlobArea', 400);

end

function tracks = initializeTracks()

tracks = struct(...

'id', {}, ...

'bbox', {}, ...

'kalmanFilter', {}, ...

'age', {}, ...

'totalVisibleCount', {}, ...

'consecutiveInvisibleCount', {});

end

function [centroids, bboxes, mask] = detectObjects(frame)

mask = obj.detector.step(frame);

mask = imopen(mask, strel('rectangle', [3,3]));

mask = imclose(mask, strel('rectangle', [15, 15]));

mask = imfill(mask, 'holes');

[~, centroids, bboxes] = obj.blobAnalyser.step(mask);

end

function predictNewLocationsOfTracks()

for i = 1:length(tracks)

bbox = tracks(i).bbox;

predictedCentroid = predict(tracks(i).kalmanFilter);

predictedCentroid = int32(predictedCentroid) - bbox(3:4) / 2;

tracks(i).bbox = [predictedCentroid, bbox(3:4)];

end

end

function [assignments, unassignedTracks, unassignedDetections] = ...

detectionToTrackAssignment()

nTracks = length(tracks);

nDetections = size(centroids, 1);

cost = zeros(nTracks, nDetections);

for i = 1:nTracks

cost(i, :) = distance(tracks(i).kalmanFilter, centroids);

end

costOfNonAssignment = 20;

[assignments, unassignedTracks, unassignedDetections] = ...

assignDetectionsToTracks(cost, costOfNonAssignment);

end

function updateAssignedTracks()

numAssignedTracks = size(assignments, 1);

for i = 1:numAssignedTracks

trackIdx = assignments(i, 1);

detectionIdx = assignments(i, 2);

centroid = centroids(detectionIdx, :);

bbox = bboxes(detectionIdx, :);

correct(tracks(trackIdx).kalmanFilter, centroid);

tracks(trackIdx).bbox = bbox;

tracks(trackIdx).age = tracks(trackIdx).age + 1;

tracks(trackIdx).totalVisibleCount = ...

tracks(trackIdx).totalVisibleCount + 1;

tracks(trackIdx).consecutiveInvisibleCount = 0;

end

end

function updateUnassignedTracks()

for i = 1:length(unassignedTracks)

ind = unassignedTracks(i);

tracks(ind).age = tracks(ind).age + 1;

tracks(ind).consecutiveInvisibleCount = ...

tracks(ind).consecutiveInvisibleCount + 1;

end

end

function deleteLostTracks()

if isempty(tracks)

return;

end

invisibleForTooLong = 20;

ageThreshold = 8;

ages = [tracks(:).age];

totalVisibleCounts = [tracks(:).totalVisibleCount];

visibility = totalVisibleCounts ./ ages;

lostInds = (ages < ageThreshold & visibility < 0.6) | ...

[tracks(:).consecutiveInvisibleCount] >= invisibleForTooLong;

tracks = tracks(~lostInds);

end

function createNewTracks()

centroids = centroids(unassignedDetections, :);

bboxes = bboxes(unassignedDetections, :);

for i = 1:size(centroids, 1)

centroid = centroids(i,:);

bbox = bboxes(i, :);

kalmanFilter = configureKalmanFilter('ConstantVelocity', ...

centroid, [200, 50], [100, 25], 100);

newTrack = struct(...

'id', nextId, ...

'bbox', bbox, ...

'kalmanFilter', kalmanFilter, ...

'age', 1, ...

'totalVisibleCount', 1, ...

'consecutiveInvisibleCount', 0);

tracks(end + 1) = newTrack;

nextId = nextId + 1;

end

end

function displayTrackingResults()

frame = im2uint8(frame);

mask = uint8(repmat(mask, [1, 1, 3])) .* 255;

minVisibleCount = 8;

if ~isempty(tracks)

reliableTrackInds = ...

[tracks(:).totalVisibleCount] > minVisibleCount;

reliableTracks = tracks(reliableTrackInds);

if ~isempty(reliableTracks)

bboxes = cat(1, reliableTracks.bbox);

ids = int32([reliableTracks(:).id]);

labels = cellstr(int2str(ids'));

predictedTrackInds = ...

[reliableTracks(:).consecutiveInvisibleCount] > 0;

isPredicted = cell(size(labels));

isPredicted(predictedTrackInds) = {' predicted'};

labels = strcat(labels, isPredicted);

frame = insertObjectAnnotation(frame, 'rectangle', ...

bboxes, labels);

mask = insertObjectAnnotation(mask, 'rectangle', ...

bboxes, labels);

end

end

obj.maskPlayer.step(mask);

obj.videoPlayer.step(frame);

end

end

Answer 1

[Image Analyst]

That does not look like a good way to find black and white regions. What you should do is to convert to hsv colorspace and threshold on s to get gray regions. Then threshold on low v and high v to get white or black regions.

hsvImage = rgb2hsv(rgbImage);
hImage = hsvImage(:, :, 1);
sImage = hsvImage(:, :, 2);
vImage = hsvImage(:, :, 3);
% Get gray regions
grayPixels = sImage < 0.1; % or whatever....
% Now get white pixels.
whitePixels = grayPixels > 0.8; % or whatever.
% Now get black pixels.
blackPixels = grayPixels < 0.1; % or whatever.
% Get a mask where BOTH black and white pixels are true.
mask = whitePixels | blackPixels;
% Get rid of small blobs 100 pixels or smaller.
mask = bwareaopen(mask, 100);
% Find centroids of what's left
props = regionprops(mask, 'Centroid');
centroids = [props.Centroid];  % [x1,y1,x2,y2,x3,y3,x4,y4,......]
% Extract into two separate arrays.
xCentroids = centroids(1:2:end);
yCentroids = centroids(2:2:end);

[Fazlay Mir]

Hi Image Analyst,

With this code, my mask is always TRUE for all values. Should the white pixels should be a bitwise or with black pixels?

Fazlay

[Image Analyst]

The last sentence does not make sense to me. If your mask is all white, then you probably need to adjust the thresholds.

Why don't you try the Color Thresholder app on the Apps tab of the tool ribbon?

[Fazlay Mir]

Hi Image Analyst,

Thank you for the response. I am also looking for black and white in an RGB image, I am trying to decode a 2-D binary pattern in various images (much like reading a QR code).

In your code, "whitePixels" refers to the top portion of the value axis on HSV and vice versa for "blackPixels". I have checked the matrix grayPixels and it has values of TRUE or FALSE (1 or 0).

My question is, why do we threshold on values of TRUE or FALSE? The result will always be TRUE or FALSE regardless of what values used for the two thresholds (white = gray > 0.8 .. etc).

Thank you for any clarification and assistance.

Fazlay

[Image Analyst]

The code does not threshold on an image with true or false values - check again. It thresholds grayImage, which has values in the range 0-255, and threshold sImage, which has continuous values in the range of 0 to 1.

[Fazlay Mir]

Hi Image Analyst,

This line below gives me a matrix of 1s and 0s.

% Get gray regions

grayPixels = sImage < 0.1; % or whatever

If that is the case, then these two lines below will just be thresholding on 1s and 0s instead of continouous values.

% Now get white pixels.

whitePixels = grayPixels > 0.8; % or whatever

% Now get black pixels.

blackPixels = grayPixels < 0.1; % or whatever

I may be doing something incorrect.

Thank you,

Fazlay

[Image Analyst]

I see what you mean now. I made a mistake. It should be something like this:

% Get gray regions
grayMask = sImage < 0.1; % or whatever....
% grayMask will be a mask of only where the image has no color - just gray.
% Now get white pixels.
whitePixels = (vImage > 0.8) & grayMask; % or whatever.
% Now get black pixels.
blackPixels = (vImage < 0.1) & grayMask; % or whatever.

[Fazlay Mir]

Thanks for your help!

[TARIK SEV]

Unrecognized function or variable 'hasFrame'.

Error in MotionBasedMultiObjectTracking (line 7)

while hasFrame(obj.reader)

how can we solve this problem

function MotionBasedMultiObjectTracking()

obj = setupSystemObjects()

tracks = initializeTracks();

nextId = 1;

while hasFrame(obj.reader)

frame = readFrame(obj.reader);

[centroids, bboxes, mask] = detectObjects(frame);

predictNewLocationsOfTracks();

[assignments, unassignedTracks, unassignedDetections] = ...

detectionToTrackAssignment();

updateAssignedTracks();

updateUnassignedTracks();

deleteLostTracks();

createNewTracks();

displayTrackingResults();

end

function obj = setupSystemObjects()

obj.reader= videoinput('winvideo',1);

obj.maskPlayer = vision.VideoPlayer('Position', [740, 400, 700, 400]);

obj.videoPlayer = vision.VideoPlayer('Position', [20, 400, 700, 400]);

obj.detector = vision.ForegroundDetector('NumGaussians', 3, ...

'NumTrainingFrames', 40, 'MinimumBackgroundRatio', 0.7);

obj.blobAnalyser = vision.BlobAnalysis('BoundingBoxOutputPort', true, ...

'AreaOutputPort', true, 'CentroidOutputPort', true, ...

'MinimumBlobArea', 400);

end

function tracks = initializeTracks()

tracks = struct(...

'id', {}, ...

'bbox', {}, ...

'kalmanFilter', {}, ...

'age', {}, ...

'totalVisibleCount', {}, ...

'consecutiveInvisibleCount', {});

end

function [centroids, bboxes, mask] = detectObjects(frame)

mask = obj.detector.step(frame);

mask = imopen(mask, strel('rectangle', [3,3]));

mask = imclose(mask, strel('rectangle', [15, 15]));

mask = imfill(mask, 'holes');

[~, centroids, bboxes] = obj.blobAnalyser.step(mask);

end

function predictNewLocationsOfTracks()

for i = 1:length(tracks)

bbox = tracks(i).bbox;

predictedCentroid = predict(tracks(i).kalmanFilter);

predictedCentroid = int32(predictedCentroid) - bbox(3:4) / 2;

tracks(i).bbox = [predictedCentroid, bbox(3:4)];

end

end

function [assignments, unassignedTracks, unassignedDetections] = ...

detectionToTrackAssignment()

nTracks = length(tracks);

nDetections = size(centroids, 1);

cost = zeros(nTracks, nDetections);

for i = 1:nTracks

cost(i, :) = distance(tracks(i).kalmanFilter, centroids);

end

costOfNonAssignment = 20;

[assignments, unassignedTracks, unassignedDetections] = ...

assignDetectionsToTracks(cost, costOfNonAssignment);

end

function updateAssignedTracks()

numAssignedTracks = size(assignments, 1);

for i = 1:numAssignedTracks

trackIdx = assignments(i, 1);

detectionIdx = assignments(i, 2);

centroid = centroids(detectionIdx, :);

bbox = bboxes(detectionIdx, :);

correct(tracks(trackIdx).kalmanFilter, centroid);

tracks(trackIdx).bbox = bbox;

tracks(trackIdx).age = tracks(trackIdx).age + 1;

tracks(trackIdx).totalVisibleCount = ...

tracks(trackIdx).totalVisibleCount + 1;

tracks(trackIdx).consecutiveInvisibleCount = 0;

end

end

function updateUnassignedTracks()

for i = 1:length(unassignedTracks)

ind = unassignedTracks(i);

tracks(ind).age = tracks(ind).age + 1;

tracks(ind).consecutiveInvisibleCount = ...

tracks(ind).consecutiveInvisibleCount + 1;

end

end

function deleteLostTracks()

if isempty(tracks)

return;

end

invisibleForTooLong = 20;

ageThreshold = 8;

ages = [tracks(:).age];

totalVisibleCounts = [tracks(:).totalVisibleCount];

visibility = totalVisibleCounts ./ ages;

lostInds = (ages < ageThreshold & visibility < 0.6) | ...

[tracks(:).consecutiveInvisibleCount] >= invisibleForTooLong;

tracks = tracks(~lostInds);

end

function createNewTracks()

centroids = centroids(unassignedDetections, :);

bboxes = bboxes(unassignedDetections, :);

for i = 1:size(centroids, 1)

centroid = centroids(i,:);

bbox = bboxes(i, :);

kalmanFilter = configureKalmanFilter('ConstantVelocity', ...

centroid, [200, 50], [100, 25], 100);

newTrack = struct(...

'id', nextId, ...

'bbox', bbox, ...

'kalmanFilter', kalmanFilter, ...

'age', 1, ...

'totalVisibleCount', 1, ...

'consecutiveInvisibleCount', 0);

tracks(end + 1) = newTrack;

nextId = nextId + 1;

end

end

function displayTrackingResults()

frame = im2uint8(frame);

mask = uint8(repmat(mask, [1, 1, 3])) .* 255;

minVisibleCount = 8;

if ~isempty(tracks)

reliableTrackInds = ...

[tracks(:).totalVisibleCount] > minVisibleCount;

reliableTracks = tracks(reliableTrackInds);

if ~isempty(reliableTracks)

bboxes = cat(1, reliableTracks.bbox);

ids = int32([reliableTracks(:).id]);

labels = cellstr(int2str(ids'));

predictedTrackInds = ...

[reliableTracks(:).consecutiveInvisibleCount] > 0;

isPredicted = cell(size(labels));

isPredicted(predictedTrackInds) = {' predicted'};

labels = strcat(labels, isPredicted);

frame = insertObjectAnnotation(frame, 'rectangle', ...

bboxes, labels);

mask = insertObjectAnnotation(mask, 'rectangle', ...

bboxes, labels);

end

end

obj.maskPlayer.step(mask);

obj.videoPlayer.step(frame);

end

end