How to deal with a large number of state and action spaces?

2 ビュー (過去 30 日間)
laha_M
laha_M 2021 年 5 月 28 日
コメント済み: laha_M 2021 年 5 月 28 日
I have a custom environment where the state is N x 3 (N in the range of thousands) and action space is N x 3 x K (K is in the range of 10s).
How to pass such a large state and action space? I know I can single out each element from this large array and pass them. But is there any easier way where it's possible to pass the array as a whole than each element? (I need this when writing my custom RL environment).
Is there any suggestion in general when dealing with a large number of actions and states? Is there any example I can follow?
  2 件のコメント
David Hill
David Hill 2021 年 5 月 28 日
Does not seem very large to me. Why can't you just pass the entire state?
laha_M
laha_M 2021 年 5 月 28 日
I can pass it to the agent. But, the problem is when writing my custom environment. All the provided examples deal with a pretty less number of states and actions.
And I am not sure of the syntaxes when writing the custom MATLAB environment that deals with the whole array (state and action).
The only example I could find that creates a custom environment --
https://in.mathworks.com/help/reinforcement-learning/ug/create-custom-matlab-environment-from-template.html
It singles out the states and actions to define the dynamics.

サインインしてコメントする。

回答 (1 件)

Emmanouil Tzorakoleftherakis
Emmanouil Tzorakoleftherakis 2021 年 5 月 28 日
Even if the NX3 inputs are scalars, I would reorganize them into an "image" and use imageInput layer for the first layer as opposed to featureInputLayer. This representation may let you handle the large input space. Nx3xK actions sound a bit too much. I am not sure what the specifics of the problem are but you may be able to reformulate the problem to reduce the dimensionality
  1 件のコメント
laha_M
laha_M 2021 年 5 月 28 日
Thanks, Emmanouil. It is a resource allocation problem.
For each job (1x3 parameters describe one job, N jobs= N x 3 dimension of the state), we need to allocate 3 types of resources from K servers/machines. (Each server has 3 types of resources, and we need to provide them efficiently to fulfill the demand). That makes action states = N x K x 3, i.e., for each job, from each server allocate 3 types of resources (their quantity).

サインインしてコメントする。

カテゴリ

Help Center および File ExchangeParallel and Cloud についてさらに検索

製品


リリース

R2021a

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by