Collect Coverage for Generated C/C++ Code in Equivalence Tests
When you run C/C++ equivalence tests using software-in-the-loop (SIL) or
processor-in-the-loop (PIL) verification with Embedded Coder®, you can collect code coverage information for the generated C/C++ code.
Collect coverage by adding an instance of the matlabtest.coder.plugins.GeneratedCodeCoveragePlugin
class to the test runner.
As the tests run, the plugin collects information about the parts of the source code that
the tests executed. You can access coverage information as a code coverage report or at the
MATLAB® command line. For more information about generated C/C++ equivalence tests,
see Generate C/C++ Code and Test for Equivalence.
You can collect these types of coverage:
Statement
Function
Decision
Condition
Modified condition/decision (MC/DC)
For more information, see Types of Coverage for Generated C/C++ Code in Equivalence Tests.
Note
You can only collect coverage for C/C++ equivalence tests on Windows® and Linux® platforms.
Write SIL and PIL Equivalence Tests
You can only collect coverage for C/C++ code generated in an equivalence test if the test uses SIL or PIL verification and builds a static library. For more information, see Execute and Verify in SIL or PIL Mode.
Suppose that you want to generate code for a function called
myMath
, which allows a user to provide two numeric inputs and an
operation string to indicate whether to add or subtract the
inputs:
function y = myMath(a,b,operation) %#codegen if operation == "add" y = a+b; elseif operation == "subtract" y = b-a; else y = []; end end
operation
argument can be a variable-sized string. It uses
parameterization to execute and verify the generated C code twice, with different inputs
each
time.classdef tMyMathSIL < matlabtest.coder.TestCase properties buildResults; end methods (TestClassSetup) function generateCode(testCase) operationSize = coder.typeof("add"); operationSize.StringLength = inf; buildInputs = {1,2,operationSize}; testCase.buildResults = build(testCase,"myMath", ... Inputs=buildInputs,Configuration="lib"); end end properties (TestParameter) runInputs = {{1,2,"add"},{1,2,"subtract"}}; end methods(Test) function testSILvsMATLAB(testCase,runInputs) executionResults = execute(testCase, ... testCase.buildResults,Inputs=runInputs); verifyExecutionMatchesMATLAB(testCase,executionResults); end end end
classdef tMyMathPIL < matlabtest.coder.TestCase properties buildResults; end methods (TestClassSetup) function generateCode(testCase) operationSize = coder.typeof("add"); operationSize.StringLength = inf; buildInputs = {1,2,operationSize}; cfg = coder.config("lib","ecoder",true); cfg.Hardware = coder.hardware("ARM Cortex-M3 (QEMU)"); cfg.VerificationMode = "PIL"; testCase.buildResults = build(testCase,"myMath", ... Inputs=buildInputs,Configuration=cfg); end end properties (TestParameter) runInputs = {{1,2,"add"},{1,2,"subtract"}}; end methods(Test) function testPILvsMATLAB(testCase,runInputs) executionResults = execute(testCase, ... testCase.buildResults,Inputs=runInputs); verifyExecutionMatchesMATLAB(testCase,executionResults); end end end
Collect Coverage
To run equivalence tests with SIL or PIL verification and collect coverage:
Define the coverage format by creating an instance of
matlab.unittest.plugins.codecoverage.CoverageResult
.Create a plugin for the generated C/C++ code by creating an instance of
matlabtest.coder.plugins.GeneratedCodeCoveragePlugin
. Specify the desired coverage type for the plugin or use the default settings, which specify statement and function coverage.Create a test runner by using the
testrunner
function.Add the code coverage plugin to the test runner by using the
addPlugin
method.Create a test suite for the equivalence tests by using the
testsuite
function.Run the tests by using the
run
method.
This example code creates a test runner with a plugin that programmatically accesses the coverage information for all types of code coverage.
import matlab.unittest.plugins.codecoverage.CoverageResult import matlabtest.coder.plugins.GeneratedCodeCoveragePlugin format = CoverageResult; plugin = GeneratedCodeCoveragePlugin(Producing=format, ... MetricLevel="mcdc"); runner = testrunner("textoutput"); addPlugin(runner,plugin);
tMyMathSIL
equivalence test class and runs the tests.
All tests
pass.suite = testsuite("tMyMathSIL.m");
run(runner,suite);
Running tMyMathSIL .. Done tMyMathSIL __________
View Coverage Results
You can view coverage results in the code coverage report or programmatically access the results at the MATLAB command line.
Generate Coverage Report
To generate an interactive HTML code coverage report, pass the Result
property of the matlab.unittest.plugins.codecoverage.CoverageResult
object to the
generateHTMLReport
method. For more information about the code
coverage report, see Generate Code Coverage Report
in Collect Code Coverage Metrics for MATLAB Source Code.
This example code generates an HTML code coverage report with name
myCoverageReport
for the coverage results collected from the
test in tMyMathSIL.m
. The code stores the report in a folder
called myArtifacts
in the current
folder.
covResult = format.Result; generateHTMLReport(covResult,"myArtifacts", ... MainFile="myCoverageReport.html");
generateStandaloneReport
method.You can also generate the coverage report in the Cobertura XML format. For more information, see Generate Cobertura Coverage Reports for Generated C Code in Equivalence Tests.
View Results Programmatically
To view the coverage results programmatically, pass the Result
property of the matlab.unittest.plugins.codecoverage.CoverageResult
object to the
coverageSummary
method.
This example code returns the statement coverage results collected from the test
in
tMyMathSIL.m
.
covResult = format.Result;
covSummary = coverageSummary(covResult,"statement")
covSummary = 5×2 33 37 0 49 38 45 0 0 0 0
See Also
Classes
matlabtest.coder.TestCase
|matlabtest.coder.plugins.GeneratedCodeCoveragePlugin
|matlab.unittest.plugins.codecoverage.CoverageReport
|matlabtest.plugins.codecoverage.StandaloneReport
|matlab.unittest.plugins.codecoverage.CoverageResult