Test Case Verification Status Distribution

Metric ID

TestCaseVerificationStatusDistribution

Description

Use this metric to count the number of tests that do not have pass/fail criteria and the number of tests that do have pass/fail criteria.

A test has pass/fail criteria if it has at least one of the following:

• at least one executed verify statement

• at least one executed temporal or logical assessment

• custom criteria that has a pass/fail status in Test Manager

• baseline criteria which determine the pass/fail criteria of the test

This metric returns the result as a distribution of the results of the TestCaseVerificationStatusDistribution metric.

Computation Details

The metric:

• Includes only tests in the project that test the model or subsystems in the unit for which you collect metric data.

• Does not count the pass/fail criteria of tests that run in software-in-the-loop (SIL) or processor-in-the-loop (PIL) mode. The metric shows these tests as Missing Pass/Fail Criteria.

Collection

To collect data for this metric:

• In the Model Testing Dashboard, in the Model Test Status section, place your cursor over the Inconclusive widget.

• Use getMetrics with the metric ID TestCaseVerificationStatusDistribution.

Collecting data for this metric loads the model file and test files and requires a Simulink^® Test™ license.

Results

For this metric, instances of metric.Result return Value as a distribution structure that contains these fields:

• BinCounts — The number of tests in each bin, returned as a vector.

• BinEdges — The outputs of the TestCaseVerificationStatus metric, returned as a vector. The outputs represent the three test verification statuses:

  • 0 — The test is missing pass/fail criteria.

  • 1 — The test has pass/fail criteria.

  • 2 — The test was not run.

Compliance Thresholds

The default compliance thresholds for this metric are:

• Compliant — 0 unit tests are missing pass/fail criteria

• Non-Compliant — 1 or more unit tests do not have pass/fail criteria

• Warning — None