Assessment Macros in Polyspace Test API for C/C++ Code

Assessments form the core of a dynamic test. In any test you might author, the basic idea is to perform an assessment after the function(s) under test complete execution. An assessment checks if an expected behavior actually occurs in practice. For example, the most common form of assessment compares an expected value with an actual value and reports if the comparison was successful. Other forms of assessment can compare certain bits only, or check for some other behavior such as if an exception is thrown (C++ only).

In the Polyspace^® Test™ API for C code, assessments are implemented as macros. For example, the following assessment, which verifies if a function func returns the integer value 0, uses the PST_VERIFY_EQ_INT macro:

PST_VERIFY_EQ_INT(func(),0);

The assessment macros are defined in the header file pstunit.h, located in polyspaceroot\polyspace\pstest\pstunit\include. Here, polyspaceroot is the Polyspace installation folder, for example, C:\Program Files\Polyspace\R2026a. To use the macros, include this header file in your tests:

#include <pstunit.h>

You typically do not need to know the exact macro definitions in the header file. You can follow the syntax and usage examples in this documentation to use the macros.

Prerequisites

This topic describes test authoring using the Polyspace Test xUnit API. To compile these tests, you are required to know some file paths in advance. For your convenience, you can define environment variables to stand for the file paths, or otherwise include the file paths in your build. For more information, see Set Up C/C++ Testing and Code Profiling Using Self-Managed Builds.

Types of Assessment Macros

`ASSERT`, `VERIFY` and `ASSUME` Assessments

In the Polyspace Test API, assessments are of three kinds:

ASSERT assessments: These assessment macros begin with PST_ASSERT. If an assessment of this kind fails, the test containing the assessment stops execution and reports a failed incomplete result. The assessment failure itself is reported as an assert failure but any subsequent assessments in the test are not executed.
VERIFY assessments: These assessment macros begin with PST_VERIFY. If an assessment of this kind fails, the test continues execution and reports the failure (and any subsequent test failure).
ASSUME assessments: These assessment macros begin with PST_ASSUME. If an assessment of this kind fails, the test containing the assessment stops execution and reports an incomplete result. The assessment failure itself is reported as an assume failure but any subsequent assessments in the test are not executed.

Use an ASSERT assessment if the assessment failure renders the remaining test meaningless. For example:

Suppose that a function returns a pointer to a structure and you want to assess the values of several fields of the structure. However, if the pointer returned is NULL, you do not need to assess the structure fields. In this case, use an ASSERT assessment to first check if the pointer returned has a NULL value and then assess the structure fields.
Suppose that a function func1 takes an integer as argument and returns another integer, and a second function func2 calls func1 and uses its return value. Suppose you write a test that assesses the return value of func1 and then performs some assessment on func2. In this case, use an ASSERT assessment on func1 so that func2 is assessed only when the assessment on func1 passes.

In some cases, even though the assessment failure renders the remaining test meaningless, you might not want the assessment failure to be counted as a failure. For instance, if a test is written under an assumption and the assumption is no longer true, you might want this to be counted differently from a failing assertion. In this case, use a PST_ASSUME statement.

In all other cases, use VERIFY assessments.

Assessments with Custom Failure Messages

If you want to print a custom failure message for a failed assessment, you can use the same macro as you normally would but append _MSG to the macro name and provide an additional argument to the new macro with the custom message.

For example, to assess the equality of two integers, you normally use the assessment:

PST_VERIFY_EQ_INT(<lhs>, <rhs>);

If you want to print a custom message when the assessment fails, you can modify the assessment to the following:

PST_VERIFY_EQ_INT_MSG(<lhs>, <rhs>, <msg>);

where <msg> is the message string that you want to print.

List of Assessment Macros

The sections below list the assessment macros available with Polyspace Test.

If you use the Polyspace as You Code™ extension in Visual Studio Code, you can start typing PST_ to see autocomplete suggestions for macros from the C/C++ xUnit API. You can hover on a suggestion to see a description of the macro, and select a suggestion to enter the macro with placeholders to be filled in. For more information on the extension, see Get Started with Polyspace as You Code (Polyspace as You Code).

General Purpose Assessments

PST_VERIFY_TRUE and Related Assessments. Use one of these assessments to verify, assert, or assume that a condition <condition> evaluates to true:

PST_VERIFY_TRUE(<condition>);
PST_ASSERT_TRUE(<condition>);
PST_ASSUME_TRUE(<condition>);

PST_VERIFY(<condition>);
PST_ASSERT(<condition>);
PST_ASSUME(<condition>);

Conversely, you can use one of these assessments to verify, assert, or assume that a condition evaluates to false:

PST_VERIFY_FALSE(<condition>);
PST_ASSERT_FALSE(<condition>);
PST_ASSUME_FALSE(<condition>);

PST_VERIFY_EQ and Related Assessments. In C++, you can compare values using assessments such as PST_VERIFY_EQ unlike C, where assessments are type-specific.

Use one of these assessment macros to compare two values <lhs> and <rhs>:

PST_VERIFY_EQ(<lhs>, <rhs>); // Verify that <lhs> is equal to <rhs>
PST_ASSERT_EQ(<lhs>, <rhs>); // Assert that <lhs> is equal to <rhs>
PST_VERIFY_NE(<lhs>, <rhs>); // Verify that <lhs> is not equal to <rhs>
PST_ASSERT_NE(<lhs>, <rhs>); // Assert that <lhs> is not equal to <rhs>
PST_VERIFY_LT(<lhs>, <rhs>); // Verify that <lhs> is less than <rhs>
PST_ASSERT_LT(<lhs>, <rhs>); // Assert that <lhs> is less than <rhs>
PST_VERIFY_GT(<lhs>, <rhs>); // Verify that <lhs> is greater than <rhs>
PST_ASSERT_GT(<lhs>, <rhs>); // Assert that <lhs> is greater than <rhs>
PST_VERIFY_LE(<lhs>, <rhs>); // Verify that <lhs> is less than or equal to <rhs> 
PST_ASSERT_LE(<lhs>, <rhs>); // Assert that <lhs> is less than or equal to <rhs>
PST_VERIFY_GE(<lhs>, <rhs>); // Verify that <lhs> is greater than or equal to <rhs> 
PST_ASSERT_GE(<lhs>, <rhs>); // Assert that <lhs> is greater than or equal to <rhs>
PST_VERIFY_EQ_APPROX (<lhs>, <rhs>, <delta>); // Verify that <lhs> is equal to <rhs> within tolerance <delta>
PST_ASSERT_EQ_APPROX (<lhs>, <rhs>, <delta>); // Assert that <lhs> is equal to <rhs> within tolerance <delta>
PST_VERIFY_NE_APPROX (<lhs>, <rhs>, <delta>); // Verify that <lhs> is not equal to <rhs> within tolerance <delta>
PST_ASSERT_NE_APPROX (<lhs>, <rhs>, <delta>); // Assert that <lhs> is not equal to <rhs> within tolerance <delta>

If the objects lhs and rhs have a user-defined type, this type must contain an implementation of the appropriate operator. For instance:

For the macro PST_VERIFY_EQ to be usable with the objects, their types must contain an implementation of operator==.
For the macro PST_VERIFY_EQ_APPROX to be usable with the objects, their types must contain an implementation of the following operators:
- operator- (both unary and binary minus)
- operator==
- operator<=
For the macro PST_VERIFY_NE_APPROX to be usable with the objects, their types must contain an implementation of the following operators:
- operator- (both unary and binary minus)
- operator>

PST_VERIFY_EACH_EQ and Related Assessments. Use one of the following macros for element-by-element comparison of the first <num_elem> elements of two arrays:

PST_VERIFY_EACH_EQ(<lhs>, <rhs>, <num_elem>);
PST_ASSERT_EACH_EQ(<lhs>, <rhs>, <num_elem>);

Similar to the previous macro, this set of macros does not require a data type specification and is supported in C++ only. Here, lhs and rhs can be addresses of array elements or iterators pointing to array elements. For example, you can compare the first two elements of std::vector variables, vec1 and vec2, using one of the following:

PST_VERIFY_EACH_EQ(vec1.begin(), vec2. begin(),2);

PST_VERIFY_EACH_EQ(&vec1[0], &vec2[0],2);

Typed Assessments

In both C and C++, you can use typed assessments to compare one value lhs with another value rhs. A typed assessment involves an assessment macro that contains a data type in the macro name. This section lists the typed assessments available with the Polyspace Test xUnit API.

PST_VERIFY_EQ_INT and Related Assessments for Integers. Use one of these macros to compare signed integers:

PST_VERIFY_EQ_INT(<lhs>, <rhs>);
PST_ASSERT_EQ_INT(<lhs>, <rhs>);
PST_VERIFY_NE_INT(<lhs>, <rhs>);
PST_ASSERT_NE_INT(<lhs>, <rhs>);
PST_VERIFY_LT_INT(<lhs>, <rhs>);
PST_ASSERT_LT_INT(<lhs>, <rhs>);
PST_VERIFY_GT_INT(<lhs>, <rhs>);
PST_ASSERT_GT_INT(<lhs>, <rhs>);
PST_VERIFY_LE_INT(<lhs>, <rhs>);
PST_ASSERT_LE_INT(<lhs>, <rhs>);
PST_VERIFY_GE_INT(<lhs>, <rhs>);
PST_ASSERT_GE_INT(<lhs>, <rhs>);

Replacing INT with UINT in the above macros leads to equivalent macros for unsigned integers:

PST_VERIFY_EQ_UINT(<lhs>, <rhs>);
PST_ASSERT_EQ_UINT(<lhs>, <rhs>);
PST_VERIFY_NE_UINT(<lhs>, <rhs>);
PST_ASSERT_NE_UINT(<lhs>, <rhs>);
PST_VERIFY_LT_UINT(<lhs>, <rhs>);
PST_ASSERT_LT_UINT(<lhs>, <rhs>);
PST_VERIFY_GT_UINT(<lhs>, <rhs>);
PST_ASSERT_GT_UINT(<lhs>, <rhs>);
PST_VERIFY_LE_UINT(<lhs>, <rhs>);
PST_ASSERT_LE_UINT(<lhs>, <rhs>);
PST_VERIFY_GE_UINT(<lhs>, <rhs>);
PST_ASSERT_GE_UINT(<lhs>, <rhs>);

PST_VERIFY_EQ_BOOL and Related Assessments. Use one of these macros to compare boolean values:

PST_VERIFY_EQ_BOOL(<lhs>, <rhs>);
PST_ASSERT_EQ_BOOL(<lhs>, <rhs>);
PST_VERIFY_NE_BOOL(<lhs>, <rhs>);
PST_ASSERT_NE_BOOL(<lhs>, <rhs>);

PST_VERIFY_EQ_APPROX_FLT and Related Assessments. Use one of the following macros to compare two floating-point values. Instead of strict equality/inequality, the macros specify equality/inequality within a tolerance value <delta>.

PST_VERIFY_EQ_APPROX_FLT(<lhs>, <rhs>, <delta>);
PST_ASSERT_EQ_APPROX_FLT(<lhs>, <rhs>, <delta>);
PST_VERIFY_NE_APPROX_FLT(<lhs>, <rhs>, <delta>);
PST_ASSERT_NE_APPROX_FLT(<lhs>, <rhs>, <delta>);
PST_VERIFY_LT_FLT(<lhs>, <rhs>);
PST_ASSERT_LT_FLT(<lhs>, <rhs>);
PST_VERIFY_GT_FLT(<lhs>, <rhs>);
PST_ASSERT_GT_FLT(<lhs>, <rhs>);
PST_VERIFY_LE_FLT(<lhs>, <rhs>);
PST_ASSERT_LE_FLT(<lhs>, <rhs>);
PST_VERIFY_GE_FLT(<lhs>, <rhs>);
PST_ASSERT_GE_FLT(<lhs>, <rhs>);

Instead of using an absolute value for tolerance, you can also use one of the following macros that compare two floating-point values using a tolerance equal to 4 Units in the Last Place (ULP-s). Unlike the previous set of macros that apply to all floating-point values, you use a different macro for ULP-based comparison of float-s versus double-s since the absolute values corresponding to 1 ULP is different for the two types:

PST_VERIFY_EQ_ULP_FLOAT(<lhs>, <rhs>);
PST_ASSERT_EQ_ULP_FLOAT(<lhs>, <rhs>);
PST_VERIFY_EQ_ULP_DOUBLE(<lhs>, <rhs>);
PST_ASSERT_EQ_ULP_DOUBLE(<lhs>, <rhs>);
PST_VERIFY_NE_ULP_FLOAT(<lhs>, <rhs>);
PST_ASSERT_NE_ULP_FLOAT(<lhs>, <rhs>);
PST_VERIFY_NE_ULP_DOUBLE(<lhs>, <rhs>);
PST_ASSERT_NE_ULP_DOUBLE(<lhs>, <rhs>);

PST_VERIFY_EQ_PTR and Related Assessments. Use one of these macros to verify or assert that two pointers point to the same object:

PST_VERIFY_EQ_PTR(<lhs>, <rhs>);
PST_ASSERT_EQ_PTR(<lhs>, <rhs>);

The macro can be also be used to verify or assert that a pointer value is NULL.

Use one of these macros to verify or assert that two pointers point to different objects:

PST_VERIFY_NE_PTR(<lhs>, <rhs>);
PST_ASSERT_NE_PTR(<lhs>, <rhs>);

PST_VERIFY_EQ_CSTR and Related Assessments. Use one of these macros to verify or assert that two C-strings are the same as each other or different from each other:

PST_VERIFY_EQ_CSTR(<lhs>, <rhs>);
PST_ASSERT_EQ_CSTR(<lhs>, <rhs>);
PST_VERIFY_NE_CSTR(<lhs>, <rhs>);
PST_ASSERT_NE_CSTR(<lhs>, <rhs>);

For example, PST_VERIFY_EQ_CSTR("foo", "foo") is a passing assessment.

Use one of these macros to verify or assert that one C-string appears strictly before or after another in lexicographical order:

PST_VERIFY_LT_CSTR(<lhs>, <rhs>);
PST_ASSERT_LT_CSTR(<lhs>, <rhs>);
PST_VERIFY_GT_CSTR(<lhs>, <rhs>);
PST_ASSERT_GT_CSTR(<lhs>, <rhs>);

For example, PST_VERIFY_GT_CSTR("foo", "bar") is a passing assessment.

Use one of these macros to verify or assert that one C-string appears before or after another in lexicographical order including the possibility that both strings are the same:

PST_VERIFY_LE_CSTR(<lhs>, <rhs>);
PST_ASSERT_LE_CSTR(<lhs>, <rhs>);
PST_VERIFY_GE_CSTR(<lhs>, <rhs>);
PST_ASSERT_GE_CSTR(<lhs>, <rhs>);

PST_VERIFY_EQ_MASK_BITS and Related Assessments. Use one of these macros to verify or assert that the unmasked bits of two bit-fields are the same as each other:

PST_VERIFY_EQ_MASK_BITS(<lhs>, <rhs>, <mask>);
PST_ASSERT_EQ_MASK_BITS(<lhs>, <rhs>, <mask>);

That is, if a certain bit of <mask> is 1, the corresponding bits of <lhs> and <rhs> must be the same.

For example, suppose the binary representation of <lhs> is 01111001, and that of <rhs> is 11010101. Although these values are not equal, if <mask> is 01010001, only the second, fourth and last bit is unmasked, and the unmasked bits are equal.

PST_VERIFY_EQ_CUSTOM and Related Assessments. Use one of the following macros to verify or assert that two objects with custom data types are equal to each other.

PST_VERIFY_EQ_CUSTOM(<lhs>, <rhs>, <funcComp>, <funcDisp>);
PST_ASSERT_EQ_CUSTOM(<lhs>, <rhs>, <funcComp>, <funcDisp>);

Here, <funcComp> is a custom comparison function that can compare two objects of this type, and <funcDisp> is a custom display function that can display their values (required to show failing values when the comparison fails).

The comparison function funcComp and display function funcDisp must follow a strict format:

A function funcComp that compares two objects must have this signature:
```
int funcComp (const void *objLeft, const void *objRight);
```
The function must behave like the strcmp function in C/C++ and return 0 if the comparison succeeds.

A function funcDisp that displays an object value must have this signature:

void funcDisp (char *buffer, const pst_size_t bufferSize, const void* objToDisplay);

Assessments on Exceptions (C++ Only)

Use one of the following macros to verify that a statement <stmt> does not throw an exception:

PST_VERIFY_NO_THROW(<stmt>);
PST_ASSERT_NO_THROW(<stmt>);

Use one of the following macros to verify that a statement <stmt> throws an exception:

PST_VERIFY_ANY_THROW(<stmt>);
PST_ASSERT_ANY_THROW(<stmt>);

Use one of the following macros to verify that a statement <stmt> throws an exception of a specific type <exception_type>:

PST_VERIFY_THROW(<stmt>, <exception_type>);
PST_ASSERT_THROW(<stmt>, <exception_type>);

Execution Status Macros

When an assertion (PST_ASSERT statement) or assumption (PST_ASSUME statement) in a test fails, the remaining statements in the same scope are not executed. Suppose a test invokes a function that has a failing assertion. Once the assertion failure is encountered, execution returns to the caller of the function. To retrieve the status of the function invocation in the caller and check if all the statements in the function were executed or if there was a failed assessment, you can use these two macros:

PST_TEST_STATUS_FAIL(): Returns 1 if the function contained a failed ASSERT or VERIFY assessment.
PST_TEST_STATUS_ABORT(): Returns 1 if the function returned early because of a failing ASSERT or ASSUME assessment.

For instance, consider the following test:

PST_TEST(mySuite, myTest) {
    func();
    // Check status of execution of func()
}

Following the invocation of func(), you can use these execution status macros in conjunction to query the execution status of func():

If func() contains a failing PST_ASSERT statement, both macros PST_TEST_STATUS_FAIL() and PST_TEST_STATUS_ABORT() return 1. You can check for a failing assertion in an if statement such as:
```
if(PST_TEST_STATUS_FAIL() == 1 && PST_TEST_STATUS_ABORT() == 1) {
   // Handle failed assertion
}
```
If func() contains a failing PST_VERIFY statement, PST_TEST_STATUS_FAIL() returns 1 but PST_TEST_STATUS_ABORT() returns 0 since all statements were executed and the function did not return early.
If func() contains a failing PST_ASSUME statement, PST_TEST_STATUS_FAIL() returns 0 but PST_TEST_STATUS_ABORT() returns 1 since the function returned early.
If func() does not contain a failing assessment of any type, both PST_TEST_STATUS_FAIL() and PST_TEST_STATUS_ABORT() return 0.

You can also implement a behavior where an assertion or assumption failure in a function called from a test leads to exiting the test entirely (instead of simply returning from the function). To define this behavior, set the test configuration macro PST_RETURN_ON_ABORT to 0 in a test configuration file:

#define PST_RETURN_ON_ABORT 0

or at the command line using a compiler flag. For instance, if you use the GCC compiler, you can add the command-line flag:

-D PST_RETURN_ON_ABORT=0

For more information on test configuration macros, see Configuration Macros in Polyspace Test API for C/C++ Code.

Assessment Macros in Polyspace Test API for C/C++ Code

Prerequisites

Types of Assessment Macros

`ASSERT`, `VERIFY` and `ASSUME` Assessments

Assessments with Custom Failure Messages

List of Assessment Macros

General Purpose Assessments

Typed Assessments

Assessments on Exceptions (C++ Only)

Execution Status Macros

See Also

Topics

Assessment Macros in Polyspace Test API for C/C++ Code

Prerequisites

Types of Assessment Macros

ASSERT, VERIFY and ASSUME Assessments

Assessments with Custom Failure Messages

List of Assessment Macros

General Purpose Assessments

Typed Assessments

Assessments on Exceptions (C++ Only)

Execution Status Macros

See Also

Topics

`ASSERT`, `VERIFY` and `ASSUME` Assessments