Hello,
I’ve incorporated an external simulation environment into Ray using this general framework. The RL framework I’ve implemented looks most like picture (4).
I’ve also managed to get a Ray Tune to work with this setup, albeit in an inefficient manner. The issue I was running into is that after each Tune trial, the Policy Server exits which causes my Policy Clients to crash and brings down the whole system. My solution is to make a request to the Policy Server for an aliveness check before making a REST API call. I have to do this before every request because I don’t know when a Tune trial will exit.
Is there a way to have the Policy Server send a message to the Policy Clients that a Ray trial has completed so the Clients can exit gracefully rather than needing to message the Server before each API call?