How to choose what features to extract from an image; make your own "bagOfFeatures"

Hello, I am currently using edited example code to idenitfy letters and faces. I am currently using bagOfFeatures but my goal is to extract the features of these images myself with only extracting 8-10 features of the images.

For the letters I have created my own dataset with ten images for each letter, all capital letters, I have attached it. I want to be able to submit an image to the code and have it come back to me with what letter it is.

For the celebrirties I have created a dataset of 9 celebrities with 15 images each, I have also attached this. I have the same goal here to submit an image to the code and have it come back to me with what celebrity it is, however if the celebrity is not a part of the dataset I want it to come back with a message that says something along the line of "Celebrity not recongnized".

Overall I want to be able to have more control over the features extracted while also increasing the accuracy of my program.

Here is my code for reference:

I have also attached my datasets and test images.

 %set directory and load images
unzip('LetterData.zip');
imds = imageDatastore('LetterData','IncludeSubfolders',true,'LabelSource','foldernames');
%%
%Now we want to know the number of data in each category
tbl = countEachLabel(imds)
%%
  [trainingSet,testSet] = splitEachLabel(imds,0.3,'randomize');
 %Then we want to see what kinds of images are being used 
 figure
montage(imds.Files(1:16:end))
%%
%Now we want to seperate so we can train
[trainingSet, validationSet] = splitEachLabel(imds, 0.6, 'randomize');
disp('Training Done');
%%
%Now we creat classification with bag of feature 
bag = bagOfFeatures(trainingSet);
img = readimage(imds, 1);
featureVector = encode(bag, img);

% Plot the histogram of visual word occurrences
figure
bar(featureVector)
title('Visual word occurrences')
xlabel('Visual word index')
ylabel('Frequency of occurrence')

%% 
%Now we create the function
categoryClassifier = trainImageCategoryClassifier(trainingSet, bag);

%%
%You can now apply the newly trained classifier to categorize new images.
img = imread('r_test.jpg');
figure
imshow(img)
[labelIdx, scores] = predict(categoryClassifier, img);
categoryClassifier.Labels(labelIdx)
%%
img = imread('a_test.png');
figure
imshow(img)
[labelIdx, scores] = predict(categoryClassifier, img);
categoryClassifier.Labels(labelIdx)
%%
img = imread('c_test.png');
figure
imshow(img)
[labelIdx, scores] = predict(categoryClassifier, img);
categoryClassifier.Labels(labelIdx)
%%
img = imread(('h_test.jpg'));
figure
imshow(img)
[labelIdx, scores] = predict(categoryClassifier, img);
categoryClassifier.Labels(labelIdx)
%%
img = imread('e_test.jpg');

figure
imshow(img)
[labelIdx, scores] = predict(categoryClassifier, img);
categoryClassifier.Labels(labelIdx)
%%
img = imread('l_test.jpg');
figure
imshow(img)
[labelIdx, scores] = predict(categoryClassifier, img);

% Display the string label
categoryClassifier.Labels(labelIdx)
%%

%Now we also want to be able to idenitfy five celbrities using MATLAB

 %set directory and load images
unzip('Celebrity.zip');
imds1 = imageDatastore('Celebrity','IncludeSubfolders',true,'LabelSource','foldernames');
%%
%Now we want to know the number of data in each category
tbl = countEachLabel(imds1)
%%
  [trainingSet,testSet] = splitEachLabel(imds1,0.3,'randomize');
 %Then we want to see what kinds of images are being used 
 figure
montage(imds1.Files(1:5:end))
%%
%Now we want to seperate so we can train
[trainingSet, validationSet] = splitEachLabel(imds1, 0.6, 'randomize');
%%
%Now we creat classification with bag of feature 
bag = bagOfFeatures(trainingSet);
img = readimage(imds1, 1);
featureVector = encode(bag, img);

% Plot the histogram of visual word occurrences
figure
bar(featureVector)
title('Visual word occurrences')
xlabel('Visual word index')
ylabel('Frequency of occurrence')

%% 
%Now we create the function
categoryClassifier = trainImageCategoryClassifier(trainingSet, bag);

%You can now apply the newly trained classifier to categorize new images.
img = imread('Brad Pitt_test.jpg');
figure
imshow(img)
[labelIdx, scores] = predict(categoryClassifier, img);
categoryClassifier.Labels(labelIdx)
%%
img = imread('Casey Musgraves_test.jpg');
figure
imshow(img)
[labelIdx, scores] = predict(categoryClassifier, img);
categoryClassifier.Labels(labelIdx)
%%
img = imread('young_bill.jpg');
figure
imshow(img)
[labelIdx, scores] = predict(categoryClassifier, img);
categoryClassifier.Labels(labelIdx)
%%
img = imread('Taylor Swift_test.jpg');
figure
imshow(img)
[labelIdx, scores] = predict(categoryClassifier, img);
categoryClassifier.Labels(labelIdx)
%%
img = imread('Oprah_test.jpg');
figure
imshow(img)
[labelIdx, scores] = predict(categoryClassifier, img);

% Display the string label
categoryClassifier.Labels(labelIdx)

Answer 1

[Harsha Priya Daggubati]

Hi,

You can always create your own feature extractor to extract features from images in 'bagOfFeatures' function. Refer to the following links for more details:

https://www.mathworks.com/help/vision/ref/bagoffeatures.html#bufbdd6-extractorFcn

https://www.mathworks.com/help/vision/ug/create-a-custom-feature-extractor.html

Hope this helps you!