verifyExecutionMatchesMATLAB

Class: matlabtest.compiler.TestCase
Namespace: matlabtest.compiler

Verify that deployed code artifact execution results match MATLAB results

Since R2023a

Syntax

verifyExecutionMatchesMATLAB(testCase,executionResults)

verifyExecutionMatchesMATLAB(testCase,executionResults,diagnostic)

verifyExecutionMatchesMATLAB(___,Name=Value)

Description

verifyExecutionMatchesMATLAB(testCase,executionResults) verifies that the execution results specified by executionResults for the deployed code artifact generated by MATLAB^® Compiler SDK™ match the execution of the MATLAB source code in the equivalence test case testCase.

verifyExecutionMatchesMATLAB(testCase,executionResults,diagnostic) returns diagnostic information specified by diagnostic.

verifyExecutionMatchesMATLAB(___,Name=Value) specifies options using one or more name-value arguments in addition to the input arguments in previous syntaxes.

Input Arguments

testCase — Test case

matlabtest.compiler.TestCase object

Test case, specified as a matlabtest.compiler.TestCase object.

executionResults — Execution results

matlabtest.compiler.results.ExecutionResults

Execution results, specified as a matlabtest.compiler.results.ExecutionResults object.

diagnostic — Failure diagnostic information

string scalar | character vector | function handle | matlab.unittest.diagnostics.Diagnostic object

Failure diagnostic information, specified as a:

• String scalar or character vector

• Function handle

• matlab.automation.diagnostics.Diagnostic object or object of one of its subclasses

Example: verifyExecutionMatchesMATLAB(testCase,executionResults,"Equivalence test failed")

Name-Value Arguments

Specify optional pairs of arguments as Name1=Value1,...,NameN=ValueN, where Name is the argument name and Value is the corresponding value. Name-value arguments must appear after other arguments, but the order of the pairs does not matter.

Example: AbsTol=0.01

AbsTol — Absolute tolerance

numeric array

Absolute tolerance, specified as a numeric array. The sizes of AbsTol and expected, where expected is the output of the MATLAB execution of the function, must be the same or compatible. See Compatible Array Sizes for Basic Operations for more information about compatible arrays.

The tolerance is applied only to values of the same data type. For an absolute tolerance to be satisfied, abs(expected-actual) <= AbsTol must be true, where actual is ExecutableOutput.

RelTol — Relative tolerance

numeric array

Relative tolerance, specified as a numeric array. The sizes of RelTol and expected, where expected is the output of the MATLAB execution of the function, must be the same or compatible. See Compatible Array Sizes for Basic Operations for more information about compatible arrays.

The tolerance is applied only to values of the same data type. For a relative tolerance to be satisfied, abs(expected-actual) <= RelTol.*abs(expected) must be true, where actual is ExecutableOutput.

Examples

Generate and Test Python Package for Equivalence

This example shows how to generate a Python^® package from MATLAB source code and test for equivalence by using MATLAB Compiler SDK.

The function makesquare generates an n-by-n matrix:

function y = makesquare(x)
y = magic(x);
end

This class definition file defines an equivalence test case that inherits from matlabtest.compiler.TestCase. The test case in the methods block defines a test case that:

1. Builds the Python package from the makesquare function

2. Executes the Python package with input set to 5

3. Verifies the execution of the Python package against the execution of the MATLAB function makesquare with the same input

classdef tDeployment < matlabtest.compiler.TestCase
    methods(Test)
        function pythonEquivalence(testCase)
            buildResults = build(testCase,"makesquare.m", ...
                "pythonPackage");
            executionResults = execute(testCase,buildResults,{5});
            verifyExecutionMatchesMATLAB(testCase,executionResults);
        end
    end
end

Run the pythonEquivalence test.

runtests("tDeployment", ...
    ProcedureName="pythonEquivalence")

Running pythonEquivalence
..
Done pythonEquivalence
__________


ans = 

  TestResult with properties:

          Name: 'tDeployment/pythonEquivalence'
        Passed: 1
        Failed: 0
    Incomplete: 0
      Duration: 93.1237
       Details: [1×1 struct]

Totals:
   1 Passed, 0 Failed, 0 Incomplete.
   93.1237 seconds testing time.

Generate and Test Python Package for Equivalence, and Return Custom Diagnostic Result

This example shows how to generate a Python package from MATLAB source code and test for equivalence by using MATLAB Compiler SDK, and return a custom diagnostic result.

The function makesquare generates an n-by-n matrix:

function y = makesquare(x)
y = magic(x);
end

This class definition file defines an equivalence test case that inherits from matlabtest.compiler.TestCase. The test case in the methods block defines a test case that:

1. Builds the Python package from the makesquare function

2. Executes the Python package with input set to 5

3. Verifies the execution of the Python package against the execution of the MATLAB function makesquare with the same input and returns a string if the test fails

classdef tDeployment < matlabtest.compiler.TestCase
    methods(Test)
        function pythonEquivalence(testCase)
            buildResults = build(testCase,"makesquare.m", ...
                "pythonPackage");
            executionResults = execute(testCase,buildResults,{5});
            d = "Equivalence test failed.";
            verifyExecutionMatchesMATLAB(testCase,executionResults,d);
        end
    end
end

Run the pythonEquivalence test.

runtests("tDeployment", ...
    ProcedureName="pythonEquivalence")

Running pythonEquivalence
..
Done pythonEquivalence
__________


ans = 

  TestResult with properties:

          Name: 'tDeployment/pythonEquivalence'
        Passed: 1
        Failed: 0
    Incomplete: 0
      Duration: 93.1237
       Details: [1×1 struct]

Totals:
   1 Passed, 0 Failed, 0 Incomplete.
   93.1237 seconds testing time.

Generate and Test Python Package for Equivalence with Additional Customizations

This example shows how to generate a Python package from MATLAB source code and test for equivalence by using MATLAB Compiler SDK.

The function makesquare generates an n-by-n matrix and the function myAdd takes two inputs and adds them together:

function y = makesquare(x)
y = magic(x);
end

function y = myAdd(a,b)
y = a + b;
end

This class definition file:

• Builds a packaged that contains makesquare and myAdd

• Saves the build results object if the test fails

• Verifies the result with an absolute tolerance

classdef tDeployment < matlabtest.compiler.TestCase
    methods(Test)
        function pythonEquivalence(testCase)
            functionsToDeploy = ["makesquare.m","myAdd.m"];
            buildResults = build(testCase,functionsToDeploy, ...
                "pythonPackage",PreservingOnFailure=true);            
            preserveOnFailure=true;
            executionResults = execute(testCase,buildResults,{5}, ...
                "makesquare",PreservingOnFailure=preserveOnFailure);
            verifyExecutionMatchesMATLAB(testCase,executionResults, ...
                AbsTol=0.0001);
        end
    end
end

Run the pythonEquivalence test.

runtests("tDeployment", ...
    ProcedureName="pythonEquivalence")

Running pythonEquivalence
..
Done pythonEquivalence
__________


ans = 

  TestResult with properties:

          Name: 'tDeployment/pythonEquivalence'
        Passed: 1
        Failed: 0
    Incomplete: 0
      Duration: 93.1237
       Details: [1×1 struct]

Totals:
   1 Passed, 0 Failed, 0 Incomplete.
   93.1237 seconds testing time.

Version History

Introduced in R2023a