Colour and Pattern Matching

Hi, I am developing a Fabric Pattern and Colour Matching System. I hope to have the right thought process on this project. The system is to find matching Database fabric image based on user’s input of random Testing fabric images. The system is as follow:

1. Database consists of a number of fabric images, of various colour & pattern (as shown in the attachment).
2. User inputs random testing images (might / might not exist in the database).
3. System finds the EXACT matching database fabric image (if available); or shows ALMOST matched database image.

I am wondering what colour space would be more suitable for this system. HSV, RGB or other colour space? Besides, what pattern recognition / feature extraction + classification methods should I use? Currently, I have planned to use normalized cross correlation for pattern recognition; however, besides this method, are there any other better pattern recognition methods?

Any links to similar/existing projects is highly appreciated as well!

[Add. Info: Full database fabric image & testing images can be found in this Google Drive link https://drive.google.com/drive/folders/0B3kRUtBpzKROamx3S1dkLUtNRFE?usp=sharing

Thank you in advance for your help!

Database Images

Answer 1

[Image Analyst]

Normalized cross correlation is good for finding where a pattern exists in an image, and how closely it matches the template. It will not be so good when the scale (magnification) is not matched up, like the pattern template is in the reference image but just has a different size, or if the pattern template is rotated. For scale and rotation invariant metrics, you might want to look at Hu's moments http://www.youtube.com/watch?v=Nc06tlZAv_Q. There are also a number of other metrics you could think up, like a list of dominant colors (see http://www.mathworks.com/matlabcentral/fileexchange/28164-color-frequency-image ) and their area fractions, certain sizes or shapes of things in the image, etc. You could build up a feature vector of a bunch of things. So you look at the overall accuracy of the things as a group rather than just using any one particular metric. This is the basis of how CBIR works, and you might investigate that field for more ideas.

As an aside, you might be interested in the color thesaurus: http://colorthesaurus.epfl.ch/

[Kent Yeap]

Hi Image Analyst, thank you for your fast reply. I have read through and tested the Color Freq Image file. However, is there a way where I can find out the dominant colors of the fabric image using HSV / LAB color space? Thank you.

[Image Analyst]

You can take histograms of the channels and try to find peaks. If you want use kmeans, you can use the attached demo. Just pass in the number of clusters (colors) you want. Or you could use rgb2ind() on the RGB image and then convert the colors to HSV or LAB color space. Or you can use PCA on the channels.

[Kent Yeap]

Thank you. Is there a way for the system to determine the number of cluster (colors) itself that is present in the image? Instead of requiring the user to pass in the cluster number. It's because the system should be an automated system and should automatically show the most matching fabric database images.

I have read through the file exchange in your profile, namely DeltaE & SimpleColorDetectionByHue as well. I am wondering if there is anything that i can adopt into my project?

Thank you.

[Image Analyst]

If you have the stats toolbox, you can get help on figuring out how many clusters to specify. Use evalclusters() or silhouette().

eva = evalclusters(x,clust,'Silhouette') % Creates a silhouette criterion clustering evaluation object.

silhouette(X,clust) plots cluster silhouettes for the n-by-p data matrix X, with clusters defined by clust. Rows of X correspond to points, columns correspond to coordinates. clust can be a categorical variable, numeric vector, character matrix, or cell array of character vectors containing a cluster name for each point. silhouette treats NaNs or empty character vectors in clust as missing values, and ignores the corresponding rows of X. By default, silhouette uses the squared Euclidean distance between points in X.

[Kent Yeap]

Hi, yup, I have the stats toolbox. Can you show me a short example of evalclusters() or silhouette() to identify the number of clusters for an image ?

I have tried these functions but got error messages.

rgbImage = imread('C1.jpeg');
% Convert RGB image to HSV
hsvImage = rgb2hsv(rgbImage);
% Extract out the H, S, and V images individually
hImage = hsvImage(:,:,1);
sImage = hsvImage(:,:,2);
vImage = hsvImage(:,:,3);
eva = evalclusters(hImage,'linkage','Silhouette'); % Creates a silhouette criterion clustering evaluation object.

Here's the error that I got:

>> evalCluster
Error using silhouette (line 82)
The number of rows in X must match the length of CLUST.

Error in evalCluster (line 9)
s = silhouette(rgbImage,'kmeans');

[Image Analyst]

Sorry, I don't really remember - it's been over a year since I used it, and that was just in a training class.

[Kent Yeap]

Hi, I see. It's okay then. Really thank you for your answer to my question!

[Image Analyst]

If we're done then, can you "Accept this answer"? Thanks in advance.

[Kent Yeap]

Hi Image Analyst, I have came around a site, http://labs.tineye.com and used their color extraction tool. The segmentation result of my image is as shown below. The system is able to segment the image based on saturation (shades) as well. I would like to develop something like that.

After researching online, here's what i think I need to do. Correct me if I am wrong, I might need to use K-mean clustering, and also have a pre-defined list of colors with hue & saturation threshold values. Then, threshold the HSV color space based on the pre-defined threshold values. Is that correct? And is there anything that I missed out?

Thank you in advance!

[Image Analyst]

Oh, so that's where that web site went to. It used to be here and I was going to give it to you but then it seemed to be defunct.

If you know how many dominant colors are in your query image, then you can use kmeans(). But what if you just have some arbitrary image with tons of colors in it? What will you use for k?

If you have a predefined list of colors in your database, then you can get the generalized linear distance of every pixel to the predefined colors. Some colors may be close while other might not be. For example what if you had a red striped cloth in your database, and a blue striped cloth, so both red and blue are in there, but then someone presents a cloth with both red and blue stripes and that patch is not in your image database. How do you handle that?

[Kent Yeap]

Hi, thank you for your fast reply. The link I posted just now is inaccurate , the correct one should be this one. http://labs.tineye.com/color/

Yeah, you are right. Kmeans and pre-defined list would not be suitable if the testing images consists of lots of different & undefined colors.

If a user were to present a cloth which does not exist in the database, the system would just show the most similar fabric images.

I am sorry to say I am still very confused and vague on what should I do for this project, as I just ventured into MATLAB image processing. But I will try my best to learn. Are you able to provide some guideline so that I am able to work on this project? I find that my previous project description is vague, here's a clearer project description:

Both database and testing fabric images are captured by user. Database would consists of 13 Plain, 15 Checks and 12 Striped fabrics. Meanwhile, the testing images would be mostly captured from fabric which are already present in the database (Batch A) and some other unknown random fabrics (Batch B). The main aim of this project would be to find the exact matching database fabric based on the testing fabric (Batch A).

For simplicity, the system will be implemented in a closed controlled illumination environment; hence the fabric images will not be affected by different lighting environment. The camera will be placed at a fixed height above the fabric to ensure that the scale of all captured fabric will be the same. Both database images and testing images will be captured using the same camera from the same fixed height and with vertical orientation.

Here's a link to the complete updated database & testing images: https://drive.google.com/drive/folders/0B3kRUtBpzKROWmVDVWhPSG11cWc?usp=sharing

Can you give me a list of steps that I should do for this project?

Thank you.

[Image Analyst]

First I'd to an opening with imopen() to get rid of white specks in the colors. Then I'd run a vertical and horizontal edge detector and see if the image has vertical edges or both vertical and horizontal edges, or no edges. That will narrow it down to 3 classes - striped, checked, or plain. Then I'd get the mean and standard deviation of the colors in the test/query image and the database image and see which matches up best.

[Kent Yeap]

Hi, thank you. I have followed what you have said, and used the imopen() and developed the edge detector. Here's the code that I have written.

clear;
i = imread('C5.jpg');
grayImg = rgb2gray(i);
bwImage=im2bw(i);
se = strel('disk',5);
grayImageOpen = imopen(bwImage,se);

BW = edge(grayImageOpen,'Sobel',.2);
imshowpair(i,BW,'montage');
cc = bwconncomp(BW);

measurements = regionprops(cc, 'BoundingBox');
% fprintf('Found %d regions\n', cc.NumObjects);

numHorizontalRegions = 0;
numVerticalRegions = 0;

for k = 1 : cc.NumObjects
  thisBB = measurements(k).BoundingBox;
  horizAspRatio(k) = thisBB(4)/thisBB(3);
    vertiAspRatio(k) = thisBB(3)/thisBB(4);

      % check horizontal
    if (thisBB(3) >= 30) && (horizAspRatio(k) >1/40)
      numHorizontalRegions = numHorizontalRegions + 1;
      end

      % check vertical 
      if (thisBB(4) >= 30) && (vertiAspRatio(k) > 1/40)
          numVerticalRegions = numVerticalRegions + 1;
      end

end

if (numHorizontalRegions > 1) && (numVerticalRegions > 1)
    fprintf('Pattern is checks fabric \n');
end 

if (numHorizontalRegions > 1) && (numVerticalRegions == 0) || (numHorizontalRegions == 0) && (numVerticalRegions > 1)
    fprintf('Pattern is striped fabric \n');
end 

if (numHorizontalRegions == 0) && (numVerticalRegions == 0)
    fprintf('Pattern is plain fabric \n');
end

The code works well for both striped and checks fabrics. However, for plain fabric, random edges are shown on the binary image. Is there anything wrong with my code? Can you help me to have a look?

Here's a link to the complete updated database & testing images: https://drive.google.com/drive/folders/0B3kRUtBpzKROOHl5ZjM2ZGdoNE0?usp=sharing

Thank you.

[Image Analyst]

If there are edges there, they should be short/small. I suggest you threshold the edge image at a higher strength so that only really strong edges are selected. And those will probably be bigger so you can throw away small edge blobs as noise. You might try making your edge detection mask larger that way it will only find big edges, not small noise edges.

[Kent Yeap]

ALright, thank you! Oh ya, is the Sobel edge detection method suitable? As MATLAB is providing 4 different edge detection methods, namely, Sobel, Prewitt, Roberts and Canny edge detector; I am wondering which should be the suitable one for my project.

Thank you.

[Image Analyst]

Try imgradient() or use fspecial() to do a difference of Gaussians filter. Or try stdfilt().

[Kent Yeap]

Okay.. I will give them a try! I am curious: Do I need to calibrate the color images as the images taken from the webcam is slightly different from the actual fabric images?

If so, how can I calibrate the color images?

[Image Analyst]

Depends on how precise you can to get. If you just have widely different colors, then you can probably find the right swatch without calibrating. However if the colors are close and the exposure or color temperature of the illumination changes a lot, then you might want to calibrate. It involves taking a picture of a standard color chart, like the X-rite Color Checker Chart. Then using it's known LAB values to develop a transform to convert RGB into LAB. See my avatar to the left. I've posted complete directions sometime before. Search for color calibration and see if you can find it.

[Kent Yeap]

Alright, thank you so much for your advice and guidelines! I will look into all these one by one.

[Kent Yeap]

Hi Image Analyst, I would like to segment my fabric images based on its hue, saturation and value/intensity values. Which color space would be more suitable for my project? HSI or HSV color model? I understand that I (from HSI) represents Intensity while V (from HSV) represents Value and the formulas are different. But what are the main differences between these 2 color spaces and in which application should I apply one but not the other color space?

Thank you.

[Image Analyst]

You'll probably get the same results either way. So I'd use hsv because there is a built in function for that, rgb2hsv() while there isn't for HSI and HSL.

Here are visualizations of your blue stripe image in the different color spaces:

[Kent Yeap]

Alright, thank you! I will install the color inspector to compare the color spaces as well.

[Image Analyst]

It's an ImageJ plugin here: https://imagej.nih.gov/ij/plugins/color-inspector.html

MATLAB has a similar function called colorcloud() but I don't think it's quite as good (yet) as the ImageJ version.