How to set Ray Python Path

Monitoring & Debugging
1

Is there any summary to discuss the python path and how it is changed/configured in Ray?

A concrete question:

  • set up python path, then using ray.init() to start a new cluster, works

  • ray start first, and then set up python path and ray.init(address=“auto”), will report module not found error.

  • setup python path, ray start, then ray.init(address = “auto”), works, but complains some weird permission issue related to a file ‘gpu_usage.dat’ (might not related to ray).

It looks like connecting to an existing ray cluster have some pitfalls in terms of python env.

Any suggested way to set up python path?

2

The third one makes the most sense to me.

Python path is propagated from the ray start command to all workers.

For example:

ray.init() # -> grabs the current env, starts workers, propagates current env to workers

$ ray start # -> grabs the current env, starts workers, propagates current env to workers

ray.init(address="auto") # -> does not start workers nor propagate current env, iiuc
1 Like