How to get last observations for the last action taken in an episode before a new episode begins

Hi Everyone,

I am new to matlab and simulink.I am using matlab simulink for doing reinforcement learning, i want to intialize my enviornment with the last set of observations (when isdone condition is true) which basically means that my enviorment do not start from a new state or a random state but it starts from the point where the last observation was taken before the new episode begins. Can someone kindly guide me how to do that?

For Example:

env.ResetFcn = @(in)setVariable(in,getObservations,'Workspace',mdl);

here i put getObservation(last observations) just for explaining.

Thanks in advance!

Answer 1

[Aditya]

To achieve the behavior you're describing in a custom reinforcement learning environment in MATLAB Simulink, where the environment starts from the last set of observations when a new episode begins instead of resetting to a new or random state, you'll need to implement some custom logic in your environment's reset function. The reset function is called at the beginning of each new episode, including the very first episode.

Here's a general approach to implement this:

1. Store the Last Observations: You need a mechanism to store the last observations whenever the isdone condition is met. This can be done within the step function of your environment.
2. Custom Reset Function: Modify the reset function to initialize the environment with these stored observations instead of resetting to a default or random state.