Obtain Coverage Report to Publish on Jira

Hello All I am trying to obtain Coverage report which can be published on JIRA. I would like to use test manager file (.mldatx) as all my tests are there. For that purpose I was using a blog Coverage Matlab and Jira. The issue I faced is that there is no way to add a plugin for .mldatx file in CodeCoveragePlugin. Below is my code

% Run tests======================================
% Import Plugins
% Import Plugins
import matlab.unittest.TestRunner;
import matlab.unittest.TestSuite;
import matlab.unittest.plugins.TAPPlugin;
import matlab.unittest.plugins.ToFile;
import('matlab.unittest.plugins.CodeCoveragePlugin');
import('matlab.unittest.plugins.codecoverage.CoberturaFormat');

% Disable Warnings
warning off Stateflow:Runtime:TestVerificationFailed;
warning off Stateflow:cdr:VerifyDangerousComparison;

try
    % Create Test Runner
    testRunnerLocal = TestRunner.withTextOutput;

      if exist('testRunnerFile.tap', 'file')== 2
         delete('testRunnerFile.tap');
      end

      % Create a TAP plugin that sends output to the file testRunnerFile.tap.
      tapFile = fullfile(getenv('WORKSPACE'), 'testRunnerFile.tap');
      tapPlugin = TAPPlugin.producingVersion13(ToFile(tapFile));

      % Add the plugin to the test runner.
      addPlugin(testRunnerLocal,tapPlugin);

      % Add Coverage Plugin 
      coverageFile = fullfile(pwd, 'coverage.xml');

      % HERE IS THE ISSUE, I tried fronFile and fromFolder
      addPlugin(testRunnerLocal,CodeCoveragePlugin.forPackage('Test1.mldatx','Producing',CoberturaFormat(coverageFile)));

      % Creating Test Suite
      testSuite = testsuite('Test1.mldatx');

      % Run the Tests
      testResults = testRunnerLocal.run(testSuite);

      % Display results
      disp(testResults);

catch exception
    disp(getReport(exception,'extended'));
    exit(1);
end

exit(0);

I am also attaching my files with it for access. Please leave a suggestion how to go about it.

Answer 1

[Rong Mi]

Hi Abhishek, Using 2017b, we do not support to publish the model coverage report to CI server automatically. The best practice to have code coverage report for all the tests is using Test Manager. You can get the accumulated coverage automatically from all test cases.

Here is the document talking about enable collecting coverage in Test Manager. https://www.mathworks.com/help/sltest/ug/functional-testing-and-coverage-analysis.html

In the "Coverage Results section", you can click the button under "REPORT" to open the html format model coverage report. Please see the below image for the button's location.

Answer 2

[Mike Wilkinson]

Hi Abhishek,

The CodeCoveragePlugin does not accept mldatx files as sources for reporting. Are you trying to collect coverage on Simulink models being tested in your mldatx file? Do you have a license for Simulink Coverage?

Thank you,

Mike Wilkinson [e-mail]

[Abhishek Kumar]

Dear Mike

I have Coverage Licences.

Is there a possibility to collect coverage for the mldtax file all the test will be defined there and it would make sense (for me at least) to use the mldtax file to collect all the data.

[Mike Wilkinson]

Hi Abhishek,

The mldatx file in this case is just a file defining your Simulink Test content. I think what you want is to collect coverage on the models tested by the TestCases and TestIterations exercised in the Simulink Test File. This is similar to how MATLAB has source files which coverage is collected on. The sources are what is defined by the CodeCoveragePlugin. The test content is then exercised in the running of the TestSuite used by the TestRunner. For 18a, we are shipping a ModelCoveragePlugin which will create the HTML reports for the automated case of running tests using the TestRunner. We are looking at ways to integrate this workflow more with Continuous Integration (CI) workflows like Jenkins. Is this the capability you are looking for? Are you currently using a Jenkins server for tracking test reporting?

Thanks,

Mike

[Abhishek Kumar]

Dear Mike

We have currently trying to integrate the CI server into our current work flow. We are indeed, using Jenkins for it. For me the question would be, what would be the best practice to have code coverage report for all the tests (if not test manager). Could you please shine some light on this topic?