polyspace.test.TestSuiteResult Class

Namespace: polyspace.test

(Python) Review test results for a specific test suite

Since R2024b

expand all in page

Description

This Python® class contains test results for a specific C/C++ test suite.

Creation

Syntax

testSuiteResult = testResults.TestSuiteResults[testSuiteIndex]

Description

testSuiteResult = testResults.TestSuiteResults[testSuiteIndex] loads the results of test suite number testSuiteIndex:

Here, testSuiteIndex is the index of the test suite specified as an integer starting from 0.

example

Properties

expand all

The properties, PassedAssessmentCount, FailedAssessmentCount, AssessmentResults, Events, Messages, PassedAssessmentResults, FailedAssessmentResults and Exceptions are associated with assessments and other events outside a test suite configuration or test. In most cases, you do not require these properties. The remaining properties are described below.

Name of test suite.

Testing framework, specified as a string with one of these values:

  • "pstunit" for tests authored in the Polyspace® Platform user interface or using the C/C++ xUnit API.

  • "external" for tests authored using another framework.

Overall execution status of the test suite, specified as a polyspace.test.StatusInfo object with the following properties:

PropertyValuesDescription
FailedTrue or False

The value of this property is True if at least one test in the suite fails or encounters a runt-time error or exception. The value can also be True if a test, which is expected to fail, passes instead. For more information on expected failures, see Write C/C++ Tests with Known Errors or Failures to Support Test-Driven Development.

In all other cases, the value is False.

IncompleteTrue or False

The value of this property is True if at least one test in the suite does not run to completion because of an assertion or assumption failure. For more information on assertions and assumptions, see Assessment Macros in Polyspace Test API for C/C++ Code.

In all other cases, the value is False.

File containing suite definition.

Line number where suite definition begins. This is the line containing the PST_SUITE or PST_SUITE_CONFIG macro (depending on whether your test suite has a configuration or not).

Timing of suite or test execution and timestamp of execution, specified as a polyspace.test.TimingInfo object with the following properties:

PropertyValueDescription
Timestampdatetime.datetime value

Time stamp of the beginning of test suite execution, for example, 2024-Jul-04 10:38:21.

ExecutionTimeFloating-point value

Time taken for test suite execution in seconds (up to millisecond precision).

Number of tests in the test suite. This number includes passed, failed, and incomplete tests.

Number of passing tests in the test suite.

Number of failing tests in the test suite.

Number of incomplete tests in the test suite.

Result details for test cases, returned as a list of polyspace.test.TestCaseResult objects with one object per test case. This property is the primary way to drill down into details of test suite results.

For more information on the object, see polyspace.test.TestCaseResult.

Results of assessments in suite configuration (if any), specified as a polyspace.test.SuiteConfigResult object. This object covers results of assessments in the PST_SUITE_CONFIG macro.

For more information on suite configuration, see Group C/C++ Tests into Suites with Common Setup and Teardown Code.

Results of assessments in the suite setup function (if any), specified as a polyspace.test.SuiteSetupResult object. This object covers results of assessments in the function registered in the PST_SUITE_SETUP macro.

For more information on suite setup, see Group C/C++ Tests into Suites with Common Setup and Teardown Code.

Results of assessments in the suite teardown function (if any), specified as a polyspace.test.SuiteTeardownResult object. This object covers results of assessments in the function registered in the PST_TEST_TEARDOWN macro.

For more information on suite teardown, see Group C/C++ Tests into Suites with Common Setup and Teardown Code.

Examples

collapse all

This example shows to print the names of failing test suites after running tests in a project.

In general, you generate and manage Polyspace Test™ results by using classes from the polyspace.project and polyspace.test modules. Before starting, make sure you can import these modules on a Python shell or in a Python script without errors. For more information, see Set Up Python API for Polyspace.

  1. Import the required modules:

    import polyspace.project
import polyspace.test
import os

  2. Add source files and xUnit test files to a project. This example uses some example source and test files available with a Polyspace Test installation. Instead, you can use your own sources and tests.

    examples_path = os.path.join(polyspace.__install_path__, "polyspace", 
                            "examples", "pstest", "Getting_Started_Example")
polyspaceProject = polyspace.project.Project("newProject")
polyspaceProject.Code.Files.add(os.path.join(examples_path, "sources", "utils.c"))
polyspaceProject.IncludePaths.add(os.path.join(examples_path, "includes"))
polyspaceProject.Tests.Files.add(os.path.join(examples_path, "tests", "test.c"))

  3. Run the tests added to the project.

    res = polyspace.test.run(polyspaceProject)

  4. Print the names of test suites that contain at least one failing test.

    for suiteIndex in range(len(res.TestSuiteResults)):
    # Read result of test suite
    resTestSuite = res.TestSuiteResults[suiteIndex]
    if resTestSuite.Status.Failed:
        print(resTestSuite.Name)

    Alternatively, you can simply iterate through the list of test suites as follows:

    for suiteIndex in res.TestSuiteResults:
    if resTestSuite.Status.Failed:
        print(resTestSuite.Name)

    If you use the example source and test files, you should see the suite name updateStateTests printed. This suite has one failing test.

Version History

Introduced in R2024b

See Also

| |

Topics