Generate C/C++ Tests for Code Coverage Objectives

Using Polyspace^® Test™, you can generate C/C++ tests for specific objectives (instead of manually authoring tests). For instance, you can generate tests that call functions with boundary values for inputs, or tests that attempt to cover true/false outcomes of all conditions in a function (full condition coverage).

This topic shows how to generate tests that satisfy predefined code coverage objectives. For information on how to generate tests that exercise boundary values of inputs, see Generate C/C++ Tests for Boundary Values of Function Inputs.

Code Coverage Objectives

You can generate tests that attempt to satisfy one of these code coverage objectives:

• Decision coverage: For full decision coverage, the generated tests must achieve a true outcome and a false outcome for all decisions in the tested function.

  See also Decision Coverage.

• Condition coverage: For full condition coverage, the generated tests must achieve a true outcome and a false outcome for all conditions in the tested function.

  See also Condition Coverage.

• Modified condition/decision coverage or MC/DC coverage: A decision can consist of one or more conditions. For full MC/DC coverage, the generated tests must reverse the outcome of each condition in a way that this reversal independently reverses the parent decision.

  See also Modified Condition/Decision Coverage (MCDC) Coverage.

Each objective in this list includes the previous objectives. For instance, if the generated tests achieve full condition coverage, they also achieve full decision coverage.

Note the following:

• The test generation process does not consider existing tests by default. In other words, even if the current tests satisfy some of the coverage objectives, the process generates as many tests as required for full coverage, ignoring the current tests in the project. To take into account existing tests when generating new ones, see Generate Additional C/C++ Tests for Missing Code Coverage.

• The automatic test generation fills in the test inputs but leaves the assessments undefined. You have to fill the assessments after test generation.

Generate Tests for Coverage

This section describes how to generate tests for code coverage objectives starting from your source files. Alternatively, you can explore automatic test generation using a preconfigured project in the Polyspace Platform user interface. See Get Started with Polyspace Test Using Example Project.

In the Polyspace Platform user interface, after you add files to a project and pre-analyze the code, you can start generating tests. To generate tests for a function:

1. On the Projects pane, right-click the function name, below the Code node of the project, and select Generate Tests (coverage metrics).

2. Change the default options for test generation if needed and click OK.

   Option	Description
   Inputs Pointer Targets	Use these sections if you want to constrain the generated tests in some way. For instance, you can fix some of the input values while generating values for others, or fix the sizes of pointer targets. For more information, see Constrain Values and Target Sizes in Generated C/C++ Tests.
   Coverage level	Choose one of the following: Decision – The generated tests achieve full decision coverage. Condition – The generated tests achieve full condition coverage, in addition to full decision coverage. MCDC – The generated tests achieve full MC/DC coverage, in addition to full condition and decision coverage. The default value is based on the Coverage metrics option in the active Testing & Profiling configuration of your project. If Coverage metrics is set to Statement, the default value of Coverage level is Decision. Otherwise, the default value is the same as the value of Coverage metrics in the active configuration (Decision, Condition, or MCDC). Changes you make in the test generation dialog box do not impact the active Testing & Profiling configuration of your project. See also Coverage metrics (-cov-metric-level).
   Relational boundary	Generate tests for full relational boundary coverage. For each relational operator in the tested function such as == or <, the generated tests must test the operator with equal operand values and values that differ by a tolerance. The tolerance value is 1 for integers, and needs to be specified for floating-point numbers. The default value is the same as the value of Enable relational boundary coverage in the active Testing & Profiling configuration of your project. Changes you make in the test generation dialog box do not impact the active Testing & Profiling configuration of your project. See also: Enable relational boundary coverage (-cov-relbound) Relational Boundary Coverage
   Absolute tolerance	Specify an absolute tolerance value for relational boundary coverage. The tolerance value applies only to relational operators that have floating-point operands. The default value is the value of Absolute tolerance specified in the active Testing & Profiling configuration of your project. Changes you make to the Absolute tolerance in the test generation dialog box do not impact the active Testing & Profiling configuration of your project. For more information, see Absolute tolerance (-cov-relbound-abs-tol).
   Relative tolerance	Specify a relative tolerance value for relational boundary coverage. The tolerance value applies only to relational operators that have floating-point operands. The default value is the value of Relative tolerance specified in the active Testing & Profiling configuration of your project. Changes you make to the Relative tolerance in the test generation dialog box do not impact the active Testing & Profiling configuration of your project. For more information, see Relative tolerance (-cov-relbound-rel-tol).
   Restrict to function	Specify that the generated tests must achieve full coverage of decisions and conditions only in the tested function. This option is enabled by default. If you want to generate tests that achieve coverage of decisions and conditions in the function callees, disable this option.
   Test generation timeout	Specify a time interval in seconds. If tests are not generated within this time period, the test generation process comes to a stop. The default timeout is 300 seconds.
   Add tests for missing coverage	By default, the test generation process does not consider existing tests. In other words, even if the current tests satisfy some of the coverage objectives, the process generates as many tests as required for full coverage, ignoring the current tests in the project. Select this option if you want to generate additional tests after taking into account current tests. If you select this option, you have to provide the path to a coverage results file (.psprof file) based on existing tests. See Generate Additional C/C++ Tests for Missing Code Coverage.

3. Follow the progress of test generation on the Generating tests window.

   If tests are successfully generated, they appear below the Tests node of the project. The Logs pane states:

   • How many tests were generated.

   • How many test generation objectives were satisfied. For instance, if a function has one if statement and one return statement, for complete decision coverage,the test generation attempts to satisfy four objectives:

     • The function must be executed in at least one test.

     • There must be at least one test that leads to the if condition being true.

     • There must be at least one test that leads to the if condition being false.

     • The function exit must be reached in at least one test.

     The four objectives might be satisfied by four or fewer tests.

4. The automatic test generation fills in the test inputs but leaves the assessments undefined. Fill the Assessments section of each generated test. See Specify Test Assessments in Polyspace Platform User Interface.

You can now build and run the generated tests just like manually authored tests. See Build and Run Tests in Polyspace Platform User Interface.