SIL Test Status Distribution

Since R2023a

Metric ID

slcomp.sil.TestStatusDistribution

Description

This metric returns a distribution of the statuses of the software-in-the-loop (SIL) test results. A SIL test can have a status of either passed, failed, disabled, not run, or untested.

Supported Artifacts

This metric collects metric results for Units in the project. To control what the dashboard classifies as a unit, see Categorize Models in Hierarchy as Components or Units.

Computation Details

The metric:

• Includes only tests in the project that test the model or subsystems in the unit for which you collect metric data.

• Shows tests as not run if the test only ran in normal mode or processor-in-the-loop (PIL) mode.

Collection

To collect data for this metric, use getMetrics with the metric ID slcomp.sil.TestStatusDistribution.

Collecting data for this metric loads the model file and test result files and requires a Simulink^® Test™ license.

Results

For this metric, instances of metric.Result return Value as a distribution structure with these fields:

• BinCounts — A numeric vector indicating the number of tests in each status category.

• BinEdges — A cell array of descriptive labels for each test status:

  • 'Failed' — The test ran and returned a fail result.

  • 'Passed' — The test ran and returned a pass result.

  • 'Disabled' — The test is intentionally excluded from execution and will not run.

  • 'Not run' — The test has not run.

  • 'Untested' — The test ran, but no verification was performed. This occurs if no verify statements or other criteria were specified, or if the criteria were specified but not executed.

• OverallCount — The total number of tests, calculated as the sum of all elements in BinCounts.

• Ratios — A numeric vector representing the percentage of tests in each category:

  • Ratios(1) — Percentage of tests that failed.

  • Ratios(2) — Percentage of tests that passed.

  • Ratios(3) — Percentage of tests that are disabled.

  • Ratios(4) — Percentage of tests that have not run.

  • Ratios(5) — Percentage of tests that ran, but did not return a pass or fail result.

  The percentages are in decimal form. For example, if 10% of tests passed and the remaining tests have not run, Ratios returns:

  [0; 0.1000; 0; 0.9000; 0]

Compliance Thresholds

The default compliance thresholds for this metric are:

• Compliant — Each of the tests passed.

• Non-Compliant — One or more tests failed, have not run, are disabled, or are untested.

• Warning — None.