Hello! I was wondering if it’s possible to implement gSDE in Rllib? Stable Baselines has it implemented so I was wondering how easy it would be to implement it similarly in Rllib.
@cwijesundara thanks for posting this question. To implement gSDE in RLlib you might want to implement the state-dependent distribution in a TorchDistribution and write an RLModule that returns in its action_dist_inputs key the logits plus a state embedding. Make sure that the RLModule.action_dist_cls attribute holds your state-dependent TorchDistribution mentioned above (either by hard-coding or by writing a ray.rllib.core.models.catalog.Catalog).
Hi @Lars_Simon_Zehnder ! Thanks for the reply! How would I approach this if I’m using the older rllib framework that doesn’t use RLModule?
@cwijesundara For the old API stack you would instead take a look at the Exploration class that underlies all exploration algorithms in ROLlib’s old API stack. You can see many examples in the same folder that implement exploration of different complexity levels.
Please notice, however, that the new API stack will be very soon the new standard in RLlib and we will not support anymore then the old API stack.