Hi @gwisaacs2 ,
communicates to your remote environment to break out of the ray ecosystem.
Also, have a look at Sven’s post from last year to see an example of how you can interact with external environments: Reinforcement Learning with RLlib in the Unity Game Engine | by Sven Mika | Distributed Computing with Ray | Medium
The most favourable solution is one where you get external RPCs out of the picture. If that is not possible and your simulator is a clunky piece of software that you have to start by hand and then waits for incoming commands (such that it cannot be properly handled as a service by your cluster), you can also try to introduce a custom resource with ray. The custom resource would be symbolic for a 1-1 mapping from rollout workers to your remote environments. Then create your own gym-style env that requires your custom resource and have your rollout workers communicate to individual instances of your simulation.
Does this help? I am sorry there is not out of the box super easy solution to this. This might require a lot of legwork.