Create and Run a RoadRunner Test Case

You can use the Test Manager or Simulink^® Test™ API commands to create and run RoadRunner test cases. RoadRunner automates different types of agents on ego vehicles in road scenario simulations.

Note

To run a RoadRunner test case, you must have a RoadRunner and a RoadRunner Scenario license.

Set Up RoadRunner and Simulink Model

Before you can run a RoadRunner test case, set up your RoadRunner project and scenario using the RoadRunner application and verify that the scenario runs as expected. See Simulate RoadRunner Scenarios with Actors Modeled in Simulink (Automated Driving Toolbox), RoadRunner Project and Scene System (RoadRunner), and Create Simple RoadRunner Scene (RoadRunner).

You must also develop the Simulink model that you use as a behavior in RoadRunner. You also use this model as the system under test in Simulink Test.

Use the Test Manager to Run a RoadRunner Test

1. Open the Test Manager by using sltest.testmanager.view.

2. Create a test file by clicking New > Test File.

3. Create a RoadRunner test case template by clicking New > Test Case > RoadRunner Test. The RoadRunner section of the test case opens.

   

4. Set the simulation options and specify the folder and file locations.

   • Console Mode — When you select this option, the RoadRunner user interface is not displayed when the test runs.

   • Force Pacing Off — When you select this option, the RoadRunner simulation runs at its normal speed. When this option is cleared, you can tune the pace in the RoadRunner application to control the speed of the simulation.

   • RoadRunner Scenario — Full path to RoadRunner scenario, which is usually in the project folder.

     Note

     You can test only one RoadRunner scenario for each test case. To test multiple scenarios, use separate test cases or test iterations.

   • Override RoadRunner Setup Settings — When you select this option, you override the preferred path settings and the section displays the RoadRunner Project Folder and RoadRunner Installation Folder fields. When you clear this option, the test case uses the preferred path settings. For more information on how to specify preferred path settings, see roadrunnerSetup (RoadRunner). For information on specifying path settings when testing in parallel, see Executing RoadRunner Tests in Parallel.

   • RoadRunner Project Folder — Full path to the RoadRunner project folder.

   • RoadRunner Installation Folder — Full path to the RoadRunner executable folder.

5. In the System Under Test section, enter the name of the Simulink model that you attached to the ego vehicle in the RoadRunner scenario. The folder for the model must be on the MATLAB^® path.

6. In the Iterations section, you can choose whether to stop the test case execution when a test iteration failure occurs by selecting Enable stop on failure for test iterations. Use the Scripted Iterations section to set model parameters or RoadRunner scenario variables.

   This example shows two scenario variables defined in RoadRunner.

   

   This image shows how to set the values of the variables in two iterations and add the iterations to the test case.

   

   You can also define custom variables in the Scripted Iterations section and then use them in a callback. For example, this image shows how to define a custom variable to use in two iterations, then use a callback to print the variable.

   

   

7. In the Callbacks section, in addition to other variables, you can use these two RoadRunner-specific variables in RoadRunner test cases:

   • sltest_roadRunnerApp — RoadRunner application object

   • sltest_roadRunnerSim — RoadRunner simulation object

8. Use other test case sections as needed. Other test case sections work the same for RoadRunner tests as they do for other types of tests. See Specify Test Properties in the Test Manager.

9. Click Run to run the RoadRunner test. A RoadRunner session starts and runs the scenario simulation and model. A RoadRunner application session opens and then closes when each iteration is complete. If you selected Console Mode, the RoadRunner interface does not display.

   The test uses the simulation mode set in the Simulink model. You cannot override the simulation mode for a RoadRunner test case in the Test Manager. To change the simulation mode, change the mode in the Simulink model.

10. When the test finishes running, the Simulation Metadata table in the Results Summary section includes RoadRunner information.

    

11. Optionally, generate reports of the RoadRunner test specifications and test results.

Use Simulink Test API Commands to Create and Run a RoadRunner Test

For general information on using Simulink Test API commands to create a test script, see Create and Run Test Cases with Scripts. To create a RoadRunner test using the API:

1. Create a test file by using sltest.testmanager.TestFile.

2. Create a new test suite by using sltest.testmanager.TestSuite.

3. Create a RoadRunner test case by using createTestCase with the runOnPlatform="RoadRunner" input argument.

4. Specify the RoadRunner options by using these setProperty method name-value arguments:

   • RoadRunnerRunInConsoleMode

   • RoadRunnerTurnOffPacing

   • RoadRunnerProjectFolder

   • OverrideRoadRunnerDefaultSettings

   • RoadRunnerInstallationFolder

   • RoadRunnerScenarioPath

   • StopOnFailure

5. Assign the Simulink model as the system under test by using the Model name-value argument of setProperty.

6. If needed, set up test iterations by using sltest.testmanager.TestIteration. You can switch the RoadRunner scenario by using setRoadRunnerScenario. You can also define custom variables and set RoadRunner scenario variables in iterations and then use the variables in callbacks. Set the values for RoadRunner scenario variables or custom variables by using setRoadRunnerScenarioVariable and setRoadRunnerCustomVariable.

7. Define callbacks for the test case, if needed, by using PreLoadCallback, PostLoadCallback, or CleanupCallback name-value arguments of the setProperty method.

8. Run the test case.

9. Generate a results report, if needed, by using sltest.testmanager.TestResultReport.

RoadRunner Test Case Limitations

• Testing multiple Simulink agents is not supported.

• Testing MATLAB agents is not supported.

• You cannot convert between a regular test case and a RoadRunner test case.

• Multiple release testing and using the Model Slicer are not supported.

• Fast restart is not supported in the Iterations section of the Test Manager.

Executing RoadRunner Tests in Parallel

When you run a RoadRunner test in parallel:

• Stop on failure for test iterations is not supported.

• Testing on a remote cluster is not supported. For more information, see parallel.Cluster (Parallel Computing Toolbox).

• Preferred path settings only propagate to new workers. To propagate path setting changes to active workers, select Override RoadRunner Setup Settings and specify the RoadRunner Project Folder and RoadRunner Installation Folder fields. Otherwise, shut down the active workers before updating the default path settings and running the test.

For more information on testing in parallel, see Run Tests Using Parallel Execution.