Unit test retrieve failed indexes

I am using the unit test framework, when some of my tests fail I can see that matlab outputs the failed indeces.

I can see the output test results is in e.g., the variable

my_test_results.Details.DiagnosticRecord.Report:

Report: '================================================================================↵Verification failed in My test.↵↵ ----------------↵ Test Diagnostic:↵ ----------------↵ SIGNAL: not within error bounds↵↵ ---------------------↵ Framework Diagnostic:↵ ---------------------↵ verifyLessThanOrEqual failed.↵ --> Each element must be less than or equal to the maximum value.↵ ↵ Failing Indices:↵ 1198 10798 10799↵ ↵ Actual Value:↵ Array size ([3 5990]) exceeds the maximum number of displayable elements (5000). Displaying the first 100 elements:↵ ↵ Columns 1 through 49↵ ↵ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0↵ ↵ Columns 50 through 98↵ ↵ 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0↵ ↵ Columns 99 through 100↵ ↵ 0 0↵ Maximum Value (Inclusive):↵ 1.000000000000000e-06↵↵ ------------------↵ Stack Information:↵ ------------------↵

As it can be seen, only three indeces failed. I already check the output plots and I think it is OK to pass this test. So my question is:

Can I retrieve the failed indexes as a vector numeric variable so I can evaluate this failure programatically, validate the failed indices and pass the test?

Answer 1

[Aashray]

Hello Luis, I hope you are having a wonderful day!

Yes, I understand that you are using MATLAB’s Unit Test Framework and want to retrieve the indices of the data points that caused the test to fail. This can be done easily by writing a simple script to extract the indices from the diagnostic report from “results.Details.DiagnosticRecord.Report.” You may follow the steps below for more understanding:

1. Writing the test (I am using a dummy test just for demonstration)

classdef testMySignal < matlab.unittest.TestCase 
    methods (Test) 
        function testSignalIsLow(testCase) 
            signal = zeros(1, 10000);   % All zeros 
            signal([1198, 10798, 10799]) = 0.01;  % Introduce a few small nonzero values 
            
            % Test: Check all values <= 1e-6 
            testCase.verifyLessThanOrEqual(signal, 1e-6); 
        end 
    end 
end 

2.Execute the test using following command in the command window:

results = runtests('testMySignal'); 

3. Write a function in a separate function file to extract the indices:

function failingIndices = getFailingIndicesFromReport(reportStr) 
% Extract "Failing Indices" line using regular expression 
tokens = regexp(reportStr, 'Failing Indices:\s*([^\n]*)', 'tokens'); 

if ~isempty(tokens) 
    idxStr = tokens{1}{1}; 
    failingIndices = sscanf(idxStr, '%d').'; 
else 
    failingIndices = []; 
end 
end 

4. Get the failing indices by using the diagnostic report from the MATLAB command window:

reportStr = results.Details.DiagnosticRecord.Report; 
failingIndices = getFailingIndicesFromReport(reportStr); 
disp(failingIndices) 

You may refer to the attached images for further reference.