Set Up and Connect to Unity Game Engine

This example shows how to set up and connect to Unity® Game Engine using ROS Toolbox. You can use Unity® Game Engine to simulate various robotic applications such as pick-and-place workflow using a robotic manipulator. Unity® Game Engine supports display and customization of ROS message visualisations. As of R2023b, ROS Toolbox supports the Noetic Ninjemys distribution for ROS.

Install Unity and Create New Project

1. Download and install Unity Hub.

2. Launch Unity Hub and sign in with your account.

3. Install Unity Editor from Unity Hub. This page uses images of Unity Editor version 2021.3.6f1 LTS.

4. In Unity Hub, select Create project. Unity Editor opens with a new project.

Install Unity Packages

Install these packages using the package manager in Unity.

• URDF Importer— Download this package as a ZIP file from the URDF Importer GitHub repository to import the robot model from URDF files.

• ROS TCP Connector— Download this package as a ZIP file from the ROS TCP Connector GitHub repository to connect Unity with the ROS server.

Follow these steps to install the packages.

1. Extract the ZIP files containing the packages.

2. In Unity Editor from the menu, select Window > Package Manager to open the Package Manager window.

3. In the Package Manager window, click the + sign and select Add package from disk.

4. Browse and open the package.json files for the two packages. The screenshot indicates the locations of the package.json files each of the packages.

Set Up MATLAB for Unity Connection

Set Up ROS Master in MATLAB

Start the ROS master from MATLAB. Alternatively, you can connect Unity and MATLAB to a ROS master running elsewhere.

Close any existing ROS Master.

rosshutdown

Create a ROS Master.

rosinit

Run TCP Endpoint Node in MATLAB

A ROS TCP endpoint node can facilitate the communication between Unity and ROS for sending and receiving messages.

Start the ROS TCP endpoint node in MATLAB. You can use a Windows®, or Linux®, or Mac machine.

helperConnectROSToUnity;

Unity ROS Connection

Open the tool to specify the IP address of the machine running the ROS master in the MATLAB session by selecting Robotics > ROS Settings, from the Unity menu bar. For a detailed guide on setting up the ROS Connection, see The Unity Side.

Unity Setup Verification

Click the play button in Unity to simulate the scene. The console displays any critical errors in your setup. Address the errors before proceeding with the example. The image here is an example of simulation of a robotic manipulator for a robotic application.

Ensure the Unity scene is running and that the connection to MATLAB is functioning. When the Unity sceneries are playing, you will see an overlay on the Game pane. The blue and green arrows next to ROS IP indicate that there is a connection between Unity and the ROS master.

Unity Setup Check in MATLAB

answer = questdlg("Click the Play button in Unity. Select Yes once the connection is established.", ...
                  "Run Unity Scene","Yes","Yes");
switch answer
    case "Yes"
        % Continue
end

To check the connection status in MATLAB is functioning, verify if MATLAB is receiving the Unity published ROS topic.

rostopic list