This is machine translation

Translated by Microsoft
Mouseover text to see original. Click the button below to return to the English version of the page.

Note: This page has been translated by MathWorks. Click here to see
To view all translated materials including this page, select Country from the country navigator on the bottom of this page.

Texture Classification with Wavelet Image Scattering

This example shows how to classify textures using wavelet image scattering. In addition to Wavelet Toolbox™, this example also requires Parallel Computing Toolbox™ and Image Processing Toolbox™.

In a digital image, texture provides information about the spatial arrangement of color or pixel intensities. Particular spatial arrangements of color or pixel intensities correspond to different appearances and consistencies of the physical material being imaged. Texture classification and segmentation of images has a number of important application areas. A particularly important example is biomedical image analysis where normal and pathologic states are often characterized by morphological and histological characteristics which manifest as differences in texture.[4]

Wavelet Image Scattering

For classification problems, it is often useful to map the data into some alternative representation which discards irrelevant information while retaining the discriminative properties of each class. Wavelet image scattering constructs low-variance representations of images which are insensitive to translations and small deformations. Because translations and small deformations in the image do not affect class membership, scattering transform coefficients provide features from which you can build robust classification models.

Wavelet scattering works by cascading the image through a series of wavelet transforms, nonlinearities, and averaging [1][3][5]. The result of this deep feature extraction is that images in the same class are moved closer to each other in the scattering transform representation, while images belonging to different classes are moved farther apart.

KTH-TIPS

This example uses a publicly available texture database, the KTH-TIPS (Textures under varying Illumination, Pose, and Scale) image database. The KTH-TIPS dataset used in this example is the grayscale version. There are 810 images in total with 10 textures and 81 images per texture. The majority of images are 200-by-200 in size. This example assumes you have downloaded the KTH-TIPS grayscale dataset and untarred it so that the 10 texture classes are contained in separate subfolders of a common folder. Each subfolder is named for the class of textures it contains. Untarring the downloaded kth_tips_grey_200x200.tar file is sufficient to provide a top-level folder KTH_TIPS and the required subfolder structure.

Use the imageDatastore to read the data. Set the location property of the imageDatastore to the folder containing the KTH-TIPS database that you have access to.

location = 'C:\KTH_TIPS';
Imds = imageDatastore(location,'IncludeSubFolders',true,'FileExtensions','.png','LabelSource','foldernames');

Randomly select and visualize 20 images from the dataset.

numImages = 810;
perm = randperm(numImages,20);
for np = 1:20
    subplot(4,5,np)
    im = imread(Imds.Files{perm(np)});
    imagesc(im);    
    colormap gray; axis off;
end

Texture Classification

This example makes use of MATLAB™'s parallel processing capability through the tall array interface. Start the parallel pool if one is not currently running.

if isempty(gcp)
    parpool;
end

For reproducibility, set the random number generator. Shuffle the files of the KTH-TIPS dataset and split the 810 images into two randomly selected sets, one for training and one held-out set for testing. Use approximately 80% of the data for building a predictive model from the scattering transform and use the remaining 20% of the images for testing the model.

rng(100)
Imds = imageDatastore(location,'IncludeSubFolders',true,'FileExtensions','.png','LabelSource','foldernames');
Imds = shuffle(Imds);
[trainImds,testImds] = splitEachLabel(Imds,0.8);

We now have two datasets, a set consisting of 650 images for training with 65 images per texture and a dataset consisting of 160 images, 16 per texture for testing. You can verify this with the following.

countEachLabel(trainImds)
ans=10×2 table
        Label         Count
    ______________    _____

    aluminium_foil     65  
    brown_bread        65  
    corduroy           65  
    cotton             65  
    cracker            65  
    linen              65  
    orange_peel        65  
    sandpaper          65  
    sponge             65  
    styrofoam          65  

countEachLabel(testImds)
ans=10×2 table
        Label         Count
    ______________    _____

    aluminium_foil     16  
    brown_bread        16  
    corduroy           16  
    cotton             16  
    cracker            16  
    linen              16  
    orange_peel        16  
    sandpaper          16  
    sponge             16  
    styrofoam          16  

Create tall arrays for the resized images.

Ttrain = tall(trainImds);
Ttest = tall(testImds);

Create a scattering framework for an image input size of 200-by-200 with an InvarianceScale of 150. The invariance scale hyperparameter is the only one we set in this example. For the other hyperparameters of the scattering transform, use the default values.

sn = waveletScattering2('ImageSize',[200 200],'InvarianceScale',150);

To extract features for classification for each the training and test sets, use the helperScatImages_mean function. helperScatImages_mean resizes the images to a common 200-by-200 size and uses the scattering framework, sn, to obtain the feature matrix. In this case each feature matrix is 391-by-7-by-7. There are 391 scattering paths and each scattering coefficient image is 7-by-7. Finally, helperScatImages_mean obtains the mean along the 2nd and 3rd dimensions to obtain a 391 element feature vector for each image. Note that this is a significant reduction in data from 40,000 elements down to 391.

trainfeatures = cellfun(@(x)helperScatImages_mean(sn,x),Ttrain,'Uni',0);
testfeatures = cellfun(@(x)helperScatImages_mean(sn,x),Ttest,'Uni',0);

Using tall's gather capability, gather all the training and test feature vectors and concatenate them into matrices.

Trainf = gather(trainfeatures);
Evaluating tall expression using the Parallel Pool 'local':
- Pass 1 of 1: Completed in 1 min 43 sec
Evaluation completed in 1 min 43 sec
trainfeatures = cat(2,Trainf{:});
Testf = gather(testfeatures);
Evaluating tall expression using the Parallel Pool 'local':
- Pass 1 of 1: Completed in 26 sec
Evaluation completed in 26 sec
testfeatures = cat(2,Testf{:});

The previous code results in two matrices with row dimensions 391 and column dimension equal to the number of images in the training and test sets respectively. Accordingly, each column is a feature vector.

PCA Model and Prediction

This example constructs a simple classifier based on the principal components of the scattering feature vectors for each class. The classifier is implemented in the functions helperPCAModel and helperPCAClassifier. The function helperPCAModel determines the principal components for each digit class based on the scattering features. The function helperPCAClassifier classifies the held-out test data by finding the closest match (best projection) between the principal components of each test feature vector with the training set and assigning the class accordingly.

model = helperPCAModel(trainfeatures,30,trainImds.Labels);
predlabels = helperPCAClassifier(testfeatures,model);

After constructing the model and classifying the test set, determine the accuracy of the test set classification.

accuracy = sum(testImds.Labels == predlabels)./numel(testImds.Labels)*100
accuracy = 99.3750

We have achieved 99.375% correct classification, or a 0.625% error rate for the 160 images in the test set. A plot of the confusion matrix shows that our simple model misclassified one texture.

figure;
confusionchart(testImds.Labels,predlabels);

Summary

In this example, we used wavelet image scattering to create low-variance representations of textures for classification. Using the scattering transform and a simple principal components classifier, we achieved 99.375% correct classification on a held-out test set. This is comparable to state-of-the-art performance on the KTH-TIPS database.[2]

REFERENCES

[1] Bruna, J. & Mallat, S. (2013) "Invariant scattering convolutional networks", IEEE Transactions on Pattern Analysis and Machine Intelligence, 35(8), pp. 1872-1886.

[2] Hayman, E., Caputo, B., Fritz, M., & Eklundh, J.O. (2004) "On the significance of real-world conditions for material classification.", Computer Vision- ECCV 2004, pp. 253-266.

[3] Mallat, S. (2012) "Group invariant scattering", Communications on Pure and Applied Mathematics, 65, pp. 1331-1398.

[4] Pujol, O. & Radeva, P. (2005) "Supervised texture classification for intravascular tissue characterization". In Suri,J., Wilson, D.L, & Laxminaryan, S. (eds.) Handbook of Biomedical Image Analysis: Segmentation Models. Kluwer Academic.

[5] Sifre, L. & Mallat, S. (2013). "Rotation, scaling and deformation invariant scattering for texture discrimination", Proceedings / CVPR, IEEE Computer Society Conference on Computer Vision and Pattern Recognition. IEEE Computer Society Conference on Computer Vision and Pattern Recognition. 1233-1240. 10.1109/CVPR.2013.163.

See Also

Related Topics