I want to create one big ray cluster, and then whenever I feel like it, I would like to start an rllib training run with a current version of my code. This means that there can be many independent rllib runs at the same time on the same ray cluster. I do not want to use tune though.
Right now, it seems that if I connect more than 2 or 3 clients to the ray cluster (I do 2 or 3 times
ray.init(address=...)), I get the error
2020-12-14 12:01:08,014 WARNING services.py:202 -- Some processes that the driver needs to connect to have not registered with Redis, so retrying. Have you run 'ray start' on this node?