Creating Q-table

I have made my own RL environment where the observation is a vector of 8 elements "for example [1,2,3,4,5,6,7,8]" and the action set has four elements(["up","down","right", "left"]) and I'm trying to apply Q-Learning algorithm. my question is how to create the q-table. I did read about

T = rlTable(obsinfo,actinfo)

but I got stuck without any progress. How can I create the table?

[hsu yi cheng]

Hi, I'm working on a similar project as yours. May I have your file as a reference ? Thanks

[RUBEN HERNANDEZ]

Hi, Im' trying to simulate Q-learnig for control inverted pendulum in simulink, but i have problem to create the q-table.

For example, I have three watch signals. How can it be defined as discrete observations with their limits?

Answer 1

[Emmanouil Tzorakoleftherakis]

Did you take a look at this example? It seems to solve a similar problem.

If you want to use the provided API to create a custom grid world, I would follow this link.

If you are implemeting your own, you want to use rlFiniteSetSpec which is designed for discrete observation. rlNumericSpec is for observations continuous in nature.

[Izzat Brgli]

I did but to be honest they are not clear for me, for example :

obsInfo = rlNumericSpec([4 1]);

hier I'm defining that the obs will be a vector 4 rows and one column. But how to define my whole observations set?

[Emmanouil Tzorakoleftherakis]

If you want to use the provided API to create a custom grid world, I would follow this link.

If you are implemeting your own, you want to use rlFiniteSetSpec which is designed for discrete observation. rlNumericSpec is for observations continuous in nature.

(I updated my answer above)

[Izzat Brgli]

I did what you have recommended and it worked partially. I have right now a table which has the same row's number as the observation's number and the column's number as the action's number but without header (names for columns and rows) and the question is how can I retrieve the data or store it?

[Emmanouil Tzorakoleftherakis]

Not sure I understand the question. If you type

yourTablename.Table

you have access to all the table elements and you can use them however you need.