How to add Simulation 3D Camera onto an actor (without using Unreal Engine) visualized with 3D Scene Configuration

Hi everyone,

I am working on a mini-project to simulate the MoCap OptiTrack system in Simulink. I followed a thread on creating actors using classes and .mat files, and I can successfully view my actors (walls and a stage representing the OptiTrack room) in the 3D Scene Configuration.

However, I am struggling to correctly place the 3D cameras (Simulation 3D Cameras). If I understand the actor hierarchy correctly it seems that the "Walls" actor is the parent of "leftWall", "leftWall" is the parent of "farWall" and "farWall" is the parent of "rightwall". I am not sure if you are able to place the cameras on the "children" as I can't seem to choose a breadcrumb for its parent name, i.e. Walls/leftWall and then set a mounting offset which always seems to be shaded and inaccesible no matter the parent. Since these walls are meant to be static, I suspect I may be missing something simple.

I also tried positioning the cameras relative to the scene origin, but even then, they do not appear when I run the simulation. In the image below, I attempted to add a single camera (commented out the other 7), setting its Parent Name to the scene origin and using a Relative Translation of [1, 0, 1] as a test, just to visualize it before assigning it to its correct known location. As you can see the camera is not visible but is recognised as being part of the scene.

Thus far I have only used simulink and (besides downloading) haven't used unreal engine. I am not opposed to using it but I would prefer if we can stick to only solving this issue with simulink.

Also, as a bonus is it possible to view the capture volume of the eight cameras visually in the simulation as well?

Any advice on what I might be doing wrong or missing would be greatly appreciated. Thanks in advance!

Answer 1

[Nishan Nekoo]

Hi Shannon,

If I understand the actor hierarchy correctly it seems that the "Walls" actor is the parent of "leftWall", "leftWall" is the parent of "farWall" and "farWall" is the parent of "rightwall".

• In the picture you shared, I believe Walls is the parent of all three of the other actors (leftWall, farWall, and rightWall) as indicated by the same level of the three other actors on the indented heirarchy. Parents will have a "-" beside their name, to indicate they can be expanded to show their children.

I am not sure if you are able to place the cameras on the "children" as I can't seem to choose a breadcrumb for its parent name, i.e. Walls/leftWall and then set a mounting offset which always seems to be shaded and inaccesible no matter the parent.

• When using the "Parent name" field of the Simulation 3D Camera, you can only place cameras on Actors that exist at the top level of the model. Children of top level actors are not available. However, there is a potential workaround. You can create an empty actor at the top level of the model, call it "CameraMount" for example. On this empty actor, you can set its "Parent name" to any actor of your choice that exists in the 3D World, "leftWall", for example. Now you can attach your camera to "CameraMount" - but its effectively the same as attaching your camera directly to "leftWall", with an empty actor serving as the go-between.
• Regarding the mounting offset, this will be shaded during the simulation. It should be modifiable before the simulation. If you want to change it during the simulation, you should check the "Input" box, then add constants to the Simulink canvas that feed into the input ports. These constants can be modified while the simulation is running. Let me know if this wasn't exactly what you were referring to!

I also tried positioning the cameras relative to the scene origin, but even then, they do not appear when I run the simulation.

• When using Simulink 3D Animation, the camera is not expected to appear in the scene - it is an invisible actor (unlike in Unreal Editor where a visible camera is shown).

Thus far I have only used simulink and (besides downloading) haven't used unreal engine. I am not opposed to using it but I would prefer if we can stick to only solving this issue with simulink.

• I am confident that you can achieve your use case without using Unreal Engine and I'm happy to help you out with any issues you are facing!

Also, as a bonus is it possible to view the capture volume of the eight cameras visually in the simulation as well?

• It sounds like you want to embed the view from the cameras within the Simulation 3D Viewer window. Unfortunately, this isn't supported for now.

A few questions from me:

1. What version of MATLAB are you using?
2. To view your camera output, you should connect the "Image" output to a "Video Viewer" or "To Video Display" block. Are you already doing this? Can you share what the image looks like?
3. One way you can move this camera around interactively in the simulation is to create an empty actor, call it "CameraMount", set its mobility to sim3d.utils.MobilityTypes.Movable, then set the parent of your camera to this actor. In the Simulation 3D Viewer, you can then right click on "CameraMount", select "Edit Properties" and move it around the scene interactively by changing its translation and rotation to find the spot that you want to place it. See screenshot below.
4. Adding 8 cameras to the simulation will likely impact the simulation speed depending on what sample rates you need each camera to achieve.

I hope this helps and don't hesitate to follow up if you have additional questions.

Nishan

[Nishan Nekoo]

I had some time to create what I think you're going after, would it look something like this?

Here I've added 6 cameras, 2 on each wall.

This is what my model looked like (made in the R2025a prerelease). I am happy to share this with you depending on which version of MATLAB you are using.

We also have a light actor if you needed to add custom lighting to your scene. You can use the initialization script of the Simulation 3D Actor to add one of these to your scene.

[Shannon]

I also just wanted to check whether its possible to work with the structure from motion from multiple views (i.e using the data from each camera in an online fashion rather than taking the video collated from all the cameras in an offline fashion). The ultimate goal is to realise a 3D map/plot of the tracked motion of a markerless UAV using either object detection or image processing. Still mounting the learning curve on the theory and the technicalities of the problem but I think this would be an important issue to resolve first. Thanks

[Nishan Nekoo]

Yes this is absolutely possible. In this instance I am connecting them to a video viewer, but they can definitely be connected to other functionality available in Simulink. Here's a simple example of a video file being processed:

https://www.mathworks.com/help/vision/ug/track-marker-using-simulink-images.html

In your case, you can replace the video file with the camera input.

Here's a more complex example using Unreal:

https://www.mathworks.com/help/driving/ug/design-of-lane-marker-detector-in-3d-simulation-environment.html

Note that there is also a Simulation 3D UAV Vehicle that allows you to add a UAV to your 3D environment. If you are newer to Simulink, I would recommend the Simulink Onramp to help you get ramped up on working in Simulink.

Answer 2

[Shannon]

Hi Nishan, thank you so much for your answer it has been very informative as a newcomer to the workings of simulink. To answer your questions:

"What version of MATLAB are you using?"

I am using MATLAB R2024b.

"To view your camera output, you should connect the "Image" output to a "Video Viewer" or "To Video Display" block. Are you already doing this? Can you share what the image looks like?"

I was not doing that but I just did it with translation input [1 2 4] and I got the following image which is reassuring.

I would greatly appreciate you sharing the model you made with me if possible. Thank you so much again for all your help.

Kind Regards,

Shannon

[Nishan Nekoo]

Here's the model I created! I will warn you that having a good GPU will be necessary when running 6 camera sensors. Feel free to create a new MATLAB question if you have any other questions and accept my answer if this helped :)

Nishan