How severe does this issue affect your experience of using Ray?
- Medium: It contributes to significant difficulty to complete my task, but I can work around it.
I need to sample integers for my RL problem. In my specific domain, there are certain known heuristic values. I want to write my own custom probability distribution for sampling from gym.spaces.Box. Ideally, the sampler should be most likely to sample integers that lie close to the heuristic values, but still reserve some probability mass for random values. If I were to create a subclass of gym.spaces.Space, could it still be used with PPO without issue? In the gymnasium documentation it says “some implementations of Reinforcement Learning algorithms might not handle custom spaces properly” so I don’t know if there are other requirements to make a custom space object suitable for an action space.