Hello Ray community,
I use RLlib in combination with a custom external simulator. For this purpose, I use a PolicyServer on RLlib’s side and a client on external simulator’s side (HTTP server/client).
Now, my problem is that I cannot further speed up the simulation (i.e. faster call an env step and get an action) since communication between client and server currently takes about 100-300ms on average.
Time horizon in the env is several hours (or infinite) and in each step simulated time is incremented by 1s. Thus, episodes may still take a (too) long time.
Any recommendations on this dilemma?