Can the New Test Data be looped into the Training Data after being Classified by the CNN model?

Hello,

I am working on a CNN model but I have a limited dataset to work with. I was wondering if there is a chance that after training the CNN model on a specific number of images, when I test it against a new set of images, these new classified images can be looped into the training set. So that when the model is run again it trains with these new images as a part of the training data set.

Thanks in advance.

Answer 1

[Harsha Priya Daggubati]

Hi,

It is generally advised to keep training and test sets explicit. I can think of one possible workaround, since your dataset is of images, you can augment your dataset using 'imageDataAugmenter' and create an augmentedImageDatastore using it to train your network.

Take a look at these links for more details:

https://www.mathworks.com/help/deeplearning/ug/preprocess-images-for-deep-learning.html#mw_ef499675-d7a0-4e77-8741-ea5801695193

https://www.mathworks.com/help/deeplearning/ref/imagedataaugmenter.html

https://www.mathworks.com/help/deeplearning/ref/augmentedimagedatastore.html

Hope this helps!

[Vinay Chawla]

Thank you for responding.

The training and test datasets are explicit. What I meant by looping the data is that when I test my model with a completely new dataset after training and validation. I want that dataset to be a part of training for the next "NEW TEST" dataset.

I really hope you get what I mean.

[Harsha Priya Daggubati]

This line sounds ambiguous:

"I want that dataset to be a part of training for the next "NEW TEST" dataset"

part of training for next test set, how is it possible?

[Vinay Chawla]

That is what I am trying to figure out.

So say I train my model on 100 images and 70 are for training and 30 for validation. Afterwards, I am using a new set of images say 10, and after classifying them I want them to be added to the initial 100 images used for the model with some lines of coding so that my training data expands after each trial.

Does that make sense?

[Harsha Priya Daggubati]

So, you are thinking of adding correctly classified test set data to your traning data and thus use the resulting training dataset to train your model again.

I guess you can go ahead this way too, what is the issue you are facing?

[Vinay Chawla]

There is no issue but my study is to classify pavement distresses and the more and varied images I have, the more features the model will learn, because they occur in distinct patterns on different pavements. So,The idea is to expand the training dataset to increase learning and prediction efficiency. If I use the resulting training dataset then the model learns from the same images, correct?

[Harsha Priya Daggubati]

Yeah, to my knowledge your network would be retrained using mostly similar datasets.

[Vinay Chawla]

Okay.

Thank you for your assistance.

Appreciate it.