Editing an Aircraft Actor in Simulink-Unreal Simulation

Hello,

As part of my project, I am simulating an aircraft using Simulink, and I have set up a Simulink system to communicate with Unreal Engine (as shown in the attached image). The issue I’m facing is that when I open the Unreal project AutoVrtlEnv, which is ported from MATLAB to Unreal, I can edit the airport scene but not the aircraft itself. The aircraft is an actor class that is dynamically spawned in Unreal via the Simulation 3D Aircraft Block.

The problem is that this aircraft actor (as shown in the attached image) does not exist when the simulation is not running—it only appears once the simulation starts. While I can technically edit these dynamically spawned actors by pausing the simulation, any changes I make are lost when the simulation restarts, as the actors are recreated from scratch.

I need to modify the Sim3dMainCamera under the Sim3dMWAirliner actor class. Is there a way to modify or duplicate it so that I can edit it as a separate class without losing my changes?

Any guidance would be greatly appreciated!

[Nishan Nekoo]

Hi Oscar, could you tell me a bit more about what changes you want to make to the Sim3dMainCamera so that I can best provide advice here? I wonder if the changes you want to make can be made in Simulink itself so that you can avoid creating custom actors in Unreal.

In the meantime, I recommend taking a look at the Simulation 3D Airliner Pack block which makes it easier to create inputs for the Simulation 3D Airliner block without needing to create your own matrix concatenate. It'll make it easier if you ever wanted to use multiple inputs to the different aircraft components.

[Oscar Su]

Hi Nishan,

I'm working on implementing two modifications to the Sim3dMainCamera:

1. Adjusting the position of the internal camera view when running the simulation in VR.
2. Retrieving the VR headset's orientation in Euler angles and feeding it back into Simulink.

The reason for adjusting the internal camera view is that, when I run the simulation in Unreal's VR preview, the internal camera (as documented in the "Run Simulation" section of this) is positioned above and slightly in front of the aircraft (as shown in the attached GIF). However, in non-VR mode, it functions perfectly, remaining fixed to the aircraft's cockpit.

This issue occurs across all MathWorks-provided aerospace examples when running them in Unreal's VR preview. I have tested the following examples and observed the same behavior:

• Fly the De Havilland Beaver with Unreal Engine Visualization
• Using Unreal Engine Visualization for Airplane Flight
• Explore the NASA HL-20 Model

[Nishan Nekoo]

Hi Oscar,

I apologize I missed your response here and am just seeing it now!

Just thinking out loud here since I haven't seen this issue before.

1. When you use the VR Preview, the camera is still following along with the aircraft but just at a different offset, is that right?
2. One approach you can take to modify the Sim3dMainCamera at run time is to use Blueprints. You could try to get all actors of a certain class, and since there should only be one Sim3dMainCamera, you could then modify it at the beginning of the simulation. Have you considered an approach like this?

Nishan

[Nishan Nekoo]

I decided to try this myself and something like this allowed me to move the CameraComponent that lived under the Aircraft class and change the view that I see in the window:

It wasn't the Sim3dMainCamera that needed to be moved at run time but the CameraComponent belogning to the actor you are using (Sim3dMWAirliner in your video).

[Oscar Su]

Thank you for your response, and no worries about the delay!

1. Yes, the camera does follow the aircraft in VR preview.
2. While your suggested method moves camera positions in non-VR previews, it doesn’t affect my VR headset’s positioning, leaving the camera offset from the cockpit view.

I did found out the true cause to the issue: the VR camera is not spatially locked to the camera positions pre-defined by Simulink's Simulation 3D Scene.

And for whatever reason, the aircraft’s center of mass aligns with the lower limit of the calibrate VR play space (in this case, the floor), which meant I had to physically lie down to pilot the aircraft in its cockpit view.

Recalibrate the VR headset so that the 'floor' aligns with user's eye level is the workaround. It does create some inconvenience when using VR for other stuff, but it does midgitate the issue in this case.

- - -

With the workaround found, I have two additional questions that are the main reason I wanted to address the offset issue in the first place (apologies for not including them in the original problem statment):

1. Is it posible to anchor the VR camera to the camera positions on the Sim3DMWAirliner so it doesn't move spatially?
2. How can I send the VR headset’s view orientation (Pitch, Yaw, Roll) information in real-time back into Simulink?

Any pointers would be greatly appreciated. If this is too much for this thread, I’m happy to post a separate question.

[Nishan Nekoo]

Hi Oscar,

Thanks for the update and I am glad you figured out that it was a calibration issue with the VR Headset, I think that makes sense to me.

To your additional questions:

1. I imagine this would be possible via reparenting on initialization via blueprints - but because I don't have experience with VR Headsets, I am not sure what actors get created. If you have that information, you could try to child to the CameraComponent I mentioned in my previous comment at run time, so that their locations are locked to each other. Have you tried something like this and run into specific issues?
2. I think the Simulation 3D Message Get is what you are looking for - check out the example at that documentation link. You will need to figure out how to make the headset's view orientation available in Simulink, but I don't imagine that is too difficult once you know what class it is and use some logic similar to what I have above to get the element of that class and extract its orientation data.

I hope that helps!

Nishan