This may be a dumb question.
I often run multiple scripts that each execute an experiment/training with
tune.run. And each script requires a
ray.init. However, there may already be an existing Ray cluster in my company’s server and I’d really like to not use up unnecessary resources by starting another set of raylet, workers, etc.
Thus, I made a function that checks for any existing ray cluster, and if there is none, it starts a new one; if there is an existing cluster, it has the
address=auto setting. See below:
import ray import psutil def ray_init(): process_status = [proc for proc in psutil.process_iter() if proc.name() == "raylet"] if process_status: ray.init(include_dashboard=False, ignore_reinit_error=True, local_mode=False, address="auto") else: ray.init(include_dashboard=False, ignore_reinit_error=True, local_mode=False)
Is there a simpler/better method of doing this?