Write C/C++ Tests with Known Errors or Failures to Support Test-Driven Development

If you follow the test-driven development approach for software development, you might need to write tests even before software functionalities are fully implemented. In particular, you might be testing functions in your source code that are not fully implemented or not even defined.

This topic shows some approaches you can follow for authoring tests with known failures or compilation errors using the Polyspace^® Test™ C/C++ xUnit API.

Prerequisites

This topic describes test authoring using the Polyspace Test xUnit API. To compile these tests, you are required to know some file paths in advance. For your convenience, you can define environment variables to stand for the file paths, or otherwise include the file paths in your build. For more information, see Set Up C/C++ Testing and Code Profiling Using Self-Managed Builds.

Write Tests for Partially Implemented Functions

To write a test for a partially implemented function with known expected failures, invoke the function-like macro PST_EXPECT_FAIL() in the test body. This macro causes a test to pass as long as there is a failing assessment in the test.

For example, suppose you have to implement a function findOccurrences(uint32_t*, uint32_t) that finds the number of occurrences of a number in an array. If the function interface is defined but the function is not fully implemented, you can test the function as follows:

Invoke the macro PST_EXPECT_FAIL() in the test body.

For example, you can write a test as follows:

#include <pstunit.h>
#include <stdint.h>

// Declaration of function to test
extern uint32_t findOccurrences(uint32_t* arr, uint32_t numberToFind);

// Test data definition
uint32_t testData[] = {
    1, 0, 1, 1, 0, 0, 1
};

// Test definition
PST_SUITE(suiteFindOccurrences);
PST_TEST(suiteFindOccurrences, testFindOccurrences) {
    PST_EXPECT_FAIL();

    // Expected and actual values
    uint32_t expectedOccurrences = 5;
    uint32_t numberToFind = 1;
    uint32_t actualOccurrences;

    // Invoking function to test
    actualOccurrences = findOccurrences(testData, numberToFind);

    // Assessing function return value
    PST_VERIFY_EQ_UINT(actualOccurrences, expectedOccurrences);
}

#ifndef PSTEST_BUILD
int main(int argc, char *argv[]) {
    PST_ADD_TEST(suiteFindOccurrences, testFindOccurrences);
    return PST_MAIN(argc, argv);
}
#endif

Build the source code and test with the macro PST_ENABLE_EXPECTED_FAILURES defined at build time. For example, if you are using a GCC compiler, run the gcc command with this additional option:
```
-D PST_ENABLE_EXPECTED_FAILURES
```
Alternatively, you can add the line:
```
#define PST_ENABLE_EXPECTED_FAILURES
```
in a configuration file. For more information on configuration files, see Configuration Macros in Polyspace Test API for C/C++ Code.
As long as the function under test, findOccurrences(), is defined with the following interface and the assessment PST_VERIFY_EQ_UINT fails, this test passes even if the function is not fully implemented:
```
uint32_t findOccurrences(uint32_t* arr, uint32_t numberToFind)
```
In the test output, you see additional messages indicating that even though an assertion failed, the failure is expected. For example, if you print the output in a machine-readable format, you can see a status message expected_fail in the test status. For more information on the machine-readable format, see Machine-Readable Results for Tests Written Using Polyspace Test API.

Once the function is fully implemented, you can rerun the build without defining the macro PST_ENABLE_EXPECTED_FAILURES at build time. Any occurrence of PST_EXPECT_FAIL in the test body causes a compilation failure. You can now remove the occurrence and see if your function implementation passes your tests.

Write Tests for Functions Not Defined

If a function is not defined in your source code, invoking the function in a test causes compilation errors. You can still write tests for the function by wrapping the test body in a PST_TEST_NOT_IMPLEMENTED macro.

Suppose you have to implement a function findOccurrences to find the number of occurrences of a number in an array but you have not yet defined the function. You can test the function as follows:

Wrap the test body in a PST_TEST_NOT_IMPLEMENTED macro.

For example, you can write a test as follows:

#include <pstunit.h>
#include <stdint.h>

// Test data definition
uint32_t testData[] = {
    1, 0, 1, 1, 0, 0, 1
};

// Test definition
PST_SUITE(suiteFindOccurrences);
PST_TEST(suiteFindOccurrences, testFindOccurrences)     
     PST_TEST_NOT_IMPLEMENTED(({
        // Expected and actual values
        uint32_t expectedOccurrences = 5;
        uint32_t numberToFind = 1;

        // Invoking function under test
        uint32_t actualOccurrences = findOccurrences(testData, numberToFind);

        // Assessing function return value
        PST_VERIFY_EQ_UINT(actualOccurrences, expectedOccurrences);
    }))

#ifndef PSTEST_BUILD
int main(int argc, char *argv[]) {
    PST_ADD_TEST(suiteFindOccurrences, testFindOccurrences);
    return PST_MAIN(argc, argv);
}
#endif

Build the source code and test with the macro PST_ENABLE_EXPECTED_FAILURES defined at build time. For example, if you are using a GCC compiler, run the gcc command with this additional option:
```
-D PST_ENABLE_EXPECTED_FAILURES
```
Alternatively, you can add the line:
```
#define PST_ENABLE_EXPECTED_FAILURES
```
in a configuration file. For more information on configuration files, see Configuration Macros in Polyspace Test API for C/C++ Code.
The test compiles despite invoking an undefined function. When you execute the test, you see passing test results despite the presence of failed assertions. For example, if you print the output in a machine-readable format, you can see:
- An additional message Test is not implemented next to the assertion failure.
- A status message expected_fail in the test status.
For more information on the machine-readable format, see Machine-Readable Results for Tests Written Using Polyspace Test API.

Once the function is fully implemented, you can rerun the build without defining the macro PST_ENABLE_EXPECTED_FAILURES at build time. Any occurrence of PST_TEST_NOT_IMPLEMENTED in the test body causes a compilation failure. You can now remove the PST_TEST_NOT_IMPLEMENTED wrapper from the test body and see if your function implementation passes your tests.

Write C/C++ Tests with Known Errors or Failures to Support Test-Driven Development

Prerequisites

Write Tests for Partially Implemented Functions

Write Tests for Functions Not Defined

See Also

Topics