Main Content

Getting Started with Anomaly Detection Using Deep Learning

Anomaly detection using deep learning is an increasingly popular approach to automating visual inspection tasks. The goal of anomaly detection is to perform a binary classification of images as normal or anomalous. For example, in the manufacturing industry, anomaly detection can determine which manufactured units meet quality standards and which units are defective.

You can train an anomaly detector using semi-supervised training. A complete semi-supervised workflow consists of training a model on normal image data and determining an anomaly threshold that separates normal images from anomalous images. Anomaly detection techniques generally follow these steps:

  1. Prepare training data and calibration data.

  2. Train a model to produce anomaly scores.

  3. Calibrate the model by determining an anomaly score threshold.

  4. Classify test images using the trained model and the chosen anomaly threshold.

  5. Deploy the model onto a production line.


An alternative to anomaly detection is anomaly classification. Classification can differentiate between normal images and multiple known anomalous patterns. Classification can provide more insight on the type of anomaly, but it requires a greater amount of training data of the anomaly classes. For an example, see Classify Defects on Wafer Maps Using Deep Learning.

Prepare Training and Calibration Data

You can train an anomaly detector using one-class learning, which means that the network trains on data consisting only of normal images. Despite training on samples only of normal images, the model learns how to distinguish between normal and anomalous images. One-class learning offers many advantages for anomaly detection problems:

  • Representations of anomalies can be scarce.

  • Anomalies can represent expensive or catastrophic outcomes.

  • There can be many types of anomalies, and the types of anomalies can change over the lifetime of the model. Describing what a normal image looks like is often more feasible than providing data that represents all possible anomalies in real world settings.

Some techniques, such as Outlier Exposure, add a small number of anomalous images to the training data set.

In semi-supervised learning, you can tune the performance of the trained model using calibration data. The calibration data set consists of labeled samples of normal and anomalous images. Avoid overfitting the model by using different images in the training and calibration data sets. You can use the Image Labeler app to interactively label images with scene labels. For more information, see Get Started with the Image Labeler.

Train Model

Train a model that accepts images as input and returns scalar valued anomaly scores. The score represents the likelihood that an image is anomalous. Larger scores indicate a higher probability that an image in anomalous samples.

Optionally, you can design a network that returns localized anomaly scores in addition to a scalar aggregate score for the image. Localized score information enables you to subjectively understand the network classification decisions.

Calibrate Model

Calibrate the model by determining an anomaly score threshold that separates the scores of good samples from the scores of anomaly samples. Unfortunately, most data sets cannot be cleanly separated into these two categories, and will have misclassifications in the form of false positives (normal images classified as anomalous) and false negatives (anomalous images classified as normal). By adjusting the threshold, you can tune the false positive and false negative rates to satisfy your operating requirements.

To select a threshold, you can use statistical techniques such as receiver operating characteristic (ROC) curves. A ROC curve plots the true positive rate versus false positive rate for different thresholds. If you have Statistics and Machine Learning Toolbox™, then you can use the rocmetrics (Statistics and Machine Learning Toolbox) function to calculate metrics such as the area under the ROC curve (AUC) and accuracy.

If your model also returns localized information about anomaly scores, then you can visualize the anomaly scores to understand the classification results. For example, you can display the anomaly score map as a heatmap overlayed on a calibration image. This approach is useful for identifying patterns in false negatives and false positives. You can use these patterns to identify strategies to improve the classifier performance.

Perform Classification Using Model

Classify test images as normal or anomalous by passing the images to the trained model, then comparing the score returned by the trained model against the anomaly score threshold.

Deploy Model

The end goal of visual inspection is often to deploy trained models for use in making business decisions in real-time production systems on assembly lines. The model determines acceptance or rejection given an image. The MATLAB® Coder™ and GPU Coder™ products are effective tools for deploying visual inspection systems to CPU and NVIDIA® GPU targets.

Related Topics