Custom Autoregressive Action Models/Distributions

Hello,

I’m looking into trying out an autoregressive action model for one of my projects. I just looked through the example autoregressive model and action distribution..

The custom environment used with the autoregressive example uses an action space of (Discrete(2), Discrete(2)). The model contains a warning that it is only suitable for binary action spaces. I’m wondering if anyone can offer any insight into how the model/distribution would need to be changed to use higher dimensional discrete spaces, besides the input/output sizes in the model.

I greatly appreciate any advice!

Hi, thanks for this question!
Yeah, we should generalize this example. Looking at this briefly, I think all you have to change to make this work for higher dimensional Discrete action spaces is:

  • Remove the assertion.
  • Make a1_logits and a2_logits Dense layers the same size as the two actions, e.g. If your action space is Tuple(Discrete(4), Discrete(6)), make a1_logits size 4 and a2_logits size 6.
  • Make sure that your action distribution handles the “context” (the model’s output, which are the logits for action 1) correctly to produce an action 2.

Hope this helps, lmk.