The "createGridWorld" function is typically used to create a simple grid environment where an agent can navigate to a single goal. By default, the environment is set up to terminate the episode once the agent reaches the specified terminal state.
To train an agent to reach multiple goals in sequence within the same episode, modifying the "TerminalStates" property alone won't suffice because, as you've observed, the episode will end once the agent reaches the first terminal state. To do so, you need to implement a custom reward function and modify the environment's step logic to handle sequential goals.
- Initially, don't set any terminal states, as you want the episode to continue after reaching each goal.
- Define a reward function that provides positive reinforcement when the agent reaches one of the goals and then updates the next goal. Basically you need to create a custom reward function.
- Enhance the state representation to include the current goal or the sequence of goals that the agent needs to reach.
- Customize the step function of the environment to check if the agent has reached the current goal. When a goal is reached, update the state to reflect the next goal in the sequence(but do not terminate the episode).
- The episode should only terminate when all goals have been reached or if a maximum number of steps is exceeded.
