What happened + What you expected to happen
export RAY_ADDRESS=http://172.27.0.143:8265 && ray job submit --verbose --working-dir job – python script.py
RuntimeError: Request failed with status code 500: No available agent to submit job, please try again later..
Reproduction script
# script.py
import ray
@ray.remote
def hello_world():
return "hello world"
ray.init()
print(ray.get(hello_world.remote()))
GCS is enabled on cluster (redis)
export RAY_ADDRESS=http://172.27.0.143:8265 && ray job submit --verbose --working-dir job – python script.py
Same cluster, but without GCS works as expected job is submitted.