Main Content

HoeffdingDriftDetectionMethod

Incremental concept drift detector that utilizes Hoeffding's Bounds Drift Detection Method (HDDM)

    Description

    HoeffdingDriftDetectionMethod model object represents an incremental concept drift detector that uses the Hoeffding's Bounds nonparametric drift detection methods based on moving averages (A-test) or exponentially weighted moving averages (W-test) [1]. After creating the object, you can use the detectdrift object function to update the statistics and check for any drift in the concept data (for example, failure rate, regression loss, and so on).

    HoeffdingDriftDetectionMethod is suitable for incremental concept drift detection. For drift detection on raw data, see detectdrift for batch drift detection.

    Creation

    You can create HoeffdingDriftDetectionMethod by specifying the DetectionMethod argument as "HDDMA" or "HDDMW" in the call to incrementalConceptDriftDetector.

    Properties

    expand all

    This property is read-only.

    Type of alternative hypothesis for determining the drift status, specified as 'greater', 'less', or 'unequal'.

    Data Types: char

    This property is read-only.

    Hoeffding's bound for input data observed up to the cut point, specified as a numeric value.

    detectdrift updates CutMean and CutHoeffdingBound and resets PostCutMean and PostCutHoeffdingBound when any one of these conditions is satisfied:

    • Alternative is "greater" and Mean + HoeffdingBound is less than or equal to CutMean + CutHoeffdingBound.

    • Alternative is "less" and Mean - HoeffdingBound is greater than or equal to CutMean - CutHoeffdingBound.

    • Alternative is "unequal" and Mean + HoeffdingBound is less than or equal to CutMean - CutHoeffdingBound or Mean - HoeffdingBound is greater than or equal to CutMean + CutHoeffdingBound.

    Data Types: double

    This property is read-only.

    Weighted average of data observed up to the cut point, specified as a numeric value.

    detectdrift updates CutMean and CutHoeffdingBound and resets PostCutMean and PostCutHoeffdingBound when any one of these conditions is satisfied:

    • Alternative is "greater" and Mean + HoeffdingBound is less than or equal to CutMean + CutHoeffdingBound.

    • Alternative is "less" and Mean - HoeffdingBound is greater than or equal to CutMean - CutHoeffdingBound.

    • Alternative is "unequal" and Mean + HoeffdingBound is less than or equal to CutMean - CutHoeffdingBound or Mean - HoeffdingBound is greater than or equal to CutMean + CutHoeffdingBound.

    Data Types: double

    This property is read-only.

    Flag indicating whether software detects drift or not, specified as either 1 or 0. Value of 1 means DriftStatus is 'Drift'.

    Data Types: logical

    This property is read-only.

    Current drift status, specified as 'Stable', 'Warning', or 'Drift'. You can see the transition in the drift status by comparing DriftStatus and PreviousDriftStaus.

    Data Types: char

    This property is read-only.

    Threshold to determine if drift exists, specified as a nonnegative scalar value from 0 to 1. It is the significance level the software uses for calculating the allowed error between a random variable and its expected value in Hoeffding's inequality or McDiarmid's inequality before it sets DriftStatus to 'Drift'.

    Data Types: double

    This property is read-only.

    Number of observations used for estimating the input bound for continuous variables, specified as a nonnegative integer.

    Data Types: double

    This property is read-only.

    Note

    This option is for the exponentially weighted moving average method (ewma) only.

    Forgetting factor for the exponentially weighted moving average (EWMA) method (HDDMW), specified as a scalar value from 0 to 1.

    Data Types: double

    This property is read-only.

    Hoeffding's bound for all input data used for training the drift detector, specified as a numeric value.

    Data Types: double

    This property is read-only.

    Bounds of input data, specified as a numeric vector of size 2.

    Data Types: double

    This property is read-only.

    Type of input data, specified as either 'binary' or 'continuous'.

    Data Types: char

    This property is read-only.

    Flag indicating whether the warmup period is over or not, specified as 1 (true) or 0(false).

    Data Types: logical

    This property is read-only.

    Weighted average of all input data used for training the drift detector, specified as a numeric value.

    Data Types: double

    This property is read-only.

    Number of observations used for training the drift detector, specified as a nonnegative integer value.

    Data Types: double

    This property is read-only.

    Hoeffding's bound for data observed after the cut point, specified as a numeric value.

    detectdrift updates CutMean and CutHoeffdingBound and resets PostCutMean and PostCutHoeffdingBound when any one of these conditions is satisfied:

    • Alternative is "greater" and Mean + HoeffdingBound is less than or equal to CutMean + CutHoeffdingBound.

    • Alternative is "less" and Mean - HoeffdingBound is greater than or equal to CutMean - CutHoeffdingBound.

    • Alternative is "unequal" and Mean + HoeffdingBound is less than or equal to CutMean - CutHoeffdingBound or Mean - HoeffdingBound is greater than or equal to CutMean + CutHoeffdingBound.

    Data Types: double

    This property is read-only.

    Weighted average of data observed after the cut point, specified as a numeric value.

    detectdrift updates CutMean and CutHoeffdingBound and resets PostCutMean and PostCutHoeffdingBound when any one of these conditions is satisfied:

    • Alternative is "greater" and Mean + HoeffdingBound is less than or equal to CutMean + CutHoeffdingBound.

    • Alternative is "less" and Mean - HoeffdingBound is greater than or equal to CutMean - CutHoeffdingBound.

    • Alternative is "unequal" and Mean + HoeffdingBound is less than or equal to CutMean - CutHoeffdingBound or Mean - HoeffdingBound is greater than or equal to CutMean + CutHoeffdingBound.

    Data Types: double

    This property is read-only.

    Drift status prior to the latest training using the most recent batch of data, specified as 'Stable', 'Warning', or 'Drift'. You can see the transition in the drift status by comparing DriftStatus and PreviousDriftStaus.

    Data Types: char

    This property is read-only.

    Test method used for drift detection, specified as either 'ewma' or 'average' corresponding to the "HDDMW" and "HDDMA" detection methods, respectively, in the call to incrementalConceptDriftDetector.

    Data Types: char

    This property is read-only.

    Number of observations for drift detector warmup, specified as a nonnegative integer.

    Data Types: double

    This property is read-only.

    Flag indicating whether there is warning or not, specified as either 1 or 0. Value of 1 means DriftStatus is 'Warning'.

    Data Types: logical

    This property is read-only.

    Threshold to determine warning versus drift, specified as a nonnegative scalar value from 0 to 1. It is the significance level the software uses for calculating the allowed error between a random variable and its expected value in Hoeffding's inequality or McDiarmid's inequality before it sets DriftStatus to 'Warning'.

    Data Types: double

    Object Functions

    detectdriftUpdate drift detector states and drift status with new data
    resetReset incremental concept drift detector

    Examples

    collapse all

    Create a random stream such that the observations come from a normal distribution with standard deviation 0.75, but the mean changes over time. First 1000 observations come from a distribution with mean 2, the next 1000 come from a distribution with mean 4, and the following 1000 come from a distribution with mean 7.

    rng(1234) % For reproducibility
    numObservations = 3000;
    switchPeriod1 = 1000;
    switchPeriod2 = 2000;
    X = zeros([numObservations 1]);
    
    % Generate the data
    for i = 1:numObservations
       if i <= switchPeriod1
          X(i) = normrnd(2,0.75);
       elseif i <= switchPeriod2
          X(i) = normrnd(4,0.75);
       else
          X(i) = normrnd(7,0.75);
       end
    end

    In an incremental drift detection application, access to data stream and model update would happen consecutively. One would not collect the data first and then feed into the model. However, for the purpose of clarification, this example demonstrates the simulation of data separately.

    Specify the drift warmup period as 50 observations and estimation period for the data input bounds as 100.

    driftWarmupPeriod = 50;
    estimationPeriod = 100;

    Initiate the incremental concept drift detector. Utilize the Hoeffding's bounds method with exponential moving average method (EWMA). Specify the input type and warmup period.

    incCDDetector = incrementalConceptDriftDetector("HDDMW",InputType="continuous", ...
                    WarmupPeriod=driftWarmupPeriod,EstimationPeriod=estimationPeriod)
    incCDDetector = 
      HoeffdingDriftDetectionMethod
    
            PreviousDriftStatus: 'Stable'
                    DriftStatus: 'Stable'
                         IsWarm: 0
        NumTrainingObservations: 0
                    Alternative: 'greater'
                      InputType: 'continuous'
                     TestMethod: 'ewma'
    
    
      Properties, Methods
    
    

    incDDetector is a HoeffdingDriftDetectionMethod object. When you first create the object, properties such as DriftStatus, IsWarm, CutMean, and NumTrainingObservations are at their initial state. detectdrift updates them as you feed the data incrementally and monitor for drift.

    Preallocate the batch size and the variables to record drift status and the mean the drift detector computes with each income of data.

    status = zeros([numObservations 1]);
    statusname = strings([numObservations 1]);
    M = zeros([numObservations 1]);

    Simulate the data stream of one observation at a time and perform incremental drift detection. At each iteration:

    • Monitor for drift using the new data with detectdrift.

    • Track and record the drift status and the statistics for visualization purposes.

    • When a drift is detected, reset the incremental concept drift detector by using the function reset.

    for i = 1:numObservations
        
        incCDDetector = detectdrift(incCDDetector,X(i));
        
        M(i) = incCDDetector.Mean;
            
        if incCDDetector.DriftDetected
            status(i) = 2;
            statusname(i) = string(incCDDetector.DriftStatus);
            incCDDetector = reset(incCDDetector); % If drift detected, reset the detector
            sprintf("Drift detected at observation #%d. Detector reset.",i)
        elseif incCDDetector.WarningDetected
            status(i) = 1;
            statusname(i) = string(incCDDetector.DriftStatus);
            sprintf("Warning detected at observation #%d.",i)
        else 
            status(i) = 0;
            statusname(i) = string(incCDDetector.DriftStatus);
        end      
    end
    ans = 
    "Warning detected at observation #1024."
    
    ans = 
    "Warning detected at observation #1025."
    
    ans = 
    "Warning detected at observation #1026."
    
    ans = 
    "Warning detected at observation #1027."
    
    ans = 
    "Warning detected at observation #1028."
    
    ans = 
    "Warning detected at observation #1029."
    
    ans = 
    "Drift detected at observation #1030. Detector reset."
    
    ans = 
    "Warning detected at observation #2012."
    
    ans = 
    "Warning detected at observation #2013."
    
    ans = 
    "Warning detected at observation #2014."
    
    ans = 
    "Drift detected at observation #2015. Detector reset."
    

    Plot the drift status versus the observation number.

    gscatter(1:numObservations,status,statusname,'gyr','*',5,'on',"Number of observations","Drift status")

    Figure contains an axes object. The axes object contains 3 objects of type line. These objects represent Stable, Warning, Drift.

    Plot the mean values versus the number of observations.

    scatter(1:numObservations,M)

    Figure contains an axes object. The axes object contains an object of type scatter.

    You can see the increase in the sample mean from the plot. The mean value becomes larger and there are breaks where the software detects the drift in the mean and the you reset the incremental drift detector, hence the mean value. The observations where the sample mean is zero correspond to the estimation periods, once at the beginning and then twice after you reset the drift detector following the detection of drift.

    References

    [1] Frias-Blanco, Isvani, Jose del Campo-Ávila, Ramos-Jimenez Gonzalo, Rafael Morales-Bueno, Augustin Ortiz-Diaz, and Yaile Caballero-Mota. “Online and non-parametric drift detection methods based on Hoeffding's bounds.“ IEEE Transactions on Knowledge and Data Engineering, Vol. 27, No. 3, pp.810-823. 2014.

    Version History

    Introduced in R2022a