Discrete and Continuous actions for each step

RLlib
1

How severe does this issue affect your experience of using Ray?

  • High: It blocks me to complete my task.

Hello, I currently have a discrete action space represented by gym.spaces.Discrete(3) that represent “go right, stay and go left”. I need to add one more action, but this is a continuous value that will represent the value of acceleration or deceleration by the PPO agent.

Note that I am not looking for autoregressive actions as my actions are all independent of each other. How would I go about doing this - in my action distribution? Please advise. I appreciate it. Thank you.

EDIT
If it’s easier to make the discrete actions happen simultaneously with the continuous action, then that information would be useful too.

2

Hi @hridayns,
You would do this by adding either a Tuple or Dictionary space with a Discrete and a Box subspace.

gym.spaces.Tuple(Discrete(...), Box(...))

3

Hello, I think that’s probably sorted! I was actually wondering how that would work with the Tf Action Distribution in PPO? Do I have to make a custom distribution to handle this case or…?

4

As long as you are using the default action distributions it should be handled automatically.

5

Okay, I’ll try it and let you know how I get on. Thank you!

6

Actually, I decided not to add another action just yet. So I will bring this post to active when I try! Thank you again.