evaluate

Evaluate image classifier on collection of image sets

Syntax

confMat = evaluate(categoryClassifier,imds)

[confMat,knownLabelIdx,predictedLabelIdx,score] = evaluate(categoryClassifier,imds)

Description

confMat = evaluate(categoryClassifier,imds) returns a normalized confusion matrix, confMat.

[confMat,knownLabelIdx,predictedLabelIdx,score] = evaluate(categoryClassifier,imds) additionally returns the corresponding label indexes and score.

Examples

Train, Evaluate, and Apply Image Category Classifier

Load two image categories.

setDir  = fullfile(toolboxdir('vision'),'visiondata','imageSets');
imds = imageDatastore(setDir,'IncludeSubfolders',true,'LabelSource',...
    'foldernames');

Split the data set into a training and test data. Pick 30% of images from each set for the training data and the remainder 70% for the test data.

[trainingSet,testSet] = splitEachLabel(imds,0.3,'randomize');

Create bag of visual words.

bag = bagOfFeatures(trainingSet);

Creating Bag-Of-Features.
-------------------------
* Image category 1: books
* Image category 2: cups
* Selecting feature point locations using the Grid method.
* Extracting SURF features from the selected feature point locations.
** The GridStep is [8 8] and the BlockWidth is [32 64 96 128].

* Extracting features from 4 images...done. Extracted 76800 features.

* Keeping 80 percent of the strongest features from each category.

* Creating a 500 word visual vocabulary.
* Number of levels: 1
* Branching factor: 500
* Number of clustering steps: 1

* [Step 1/1] Clustering vocabulary level 1.
* Number of features          : 61440
* Number of clusters          : 500
* Initializing cluster centers...100.00%.
* Clustering...completed 55/100 iterations (~0.51 seconds/iteration)...converged in 55 iterations.

* Finished creating Bag-Of-Features

Train a classifier with the training sets.

categoryClassifier = trainImageCategoryClassifier(trainingSet,bag);

Training an image category classifier for 2 categories.
--------------------------------------------------------
* Category 1: books
* Category 2: cups

* Encoding features for 4 images...done.

* Finished training the category classifier. Use evaluate to test the classifier on a test set.

Evaluate the classifier using test images. Display the confusion matrix.

confMatrix = evaluate(categoryClassifier,testSet)

Evaluating image category classifier for 2 categories.
-------------------------------------------------------

* Category 1: books
* Category 2: cups

* Evaluating 8 images...done.

* Finished evaluating all the test sets.

* The confusion matrix for this test set is:


             PREDICTED
KNOWN    | books   cups   
--------------------------
books    | 0.75    0.25   
cups     | 0.25    0.75   

* Average Accuracy is 0.75.

confMatrix = 2×2

    0.7500    0.2500
    0.2500    0.7500

Find the average accuracy of the classification.

mean(diag(confMatrix))

ans = 
0.7500

Apply the newly trained classifier to categorize new images.

img = imread(fullfile(setDir,'cups','bigMug.jpg'));
[labelIdx, score] = predict(categoryClassifier,img);

Encoding images using Bag-Of-Features.
--------------------------------------
* Encoding an image...done.

Display the classification label.

categoryClassifier.Labels(labelIdx)

ans = 1×1 cell array
    {'cups'}

Input Arguments

imds — Data store object of images

ImageDatastore object

Images, specified in an ImageDatastore object.

categoryClassifier — Image category classifier

imageCategoryClassifier object

Image category classifier, specified as an imageCategoryClassifier object.

Output Arguments

confMat — Confusion matrix

matrix

Confusion matrix, returned as a matrix. The row indices correspond to known labels and the columns correspond to the predicted labels.

knownLabelIdx — Label index for image set

M-by-1 vector | scalar

Label index for image set, returned as an M-by-1 vector for M images. The knownLabelIdx output value corresponds to the index of an image set used to train the bag of features.

predictedLabelIdx — Predicted label index

M-by-1 vector

Predicted label index, returned as an M-by-1 vector for M images. The predictedLabelIdx output value corresponds to the index of an image set used to train the bag of features. The predicted index corresponds to the class with the largest value in the score output.

score — Prediction score

M-by-N matrix

Prediction score, specified as an M-by-N matrix. N represents the number of classes. M represents the number of images in the imageSet input object, imgSet. The score provides a negated average binary loss per class. Each class is a support vector machine (SVM) multiclass classifier that uses the error-correcting output codes (ECOC) approach.

Version History

Introduced in R2014b