Error in `ray job submit` on local machine if multiple clusters are running at the same time

Ray Clusters
17

Thanks, that worked for me!

  1. create test file echo 'print("Hello, World!")' >> test.py

  2. create clusters

    ray start --head --port=45521 \
  --dashboard-port=40925 \
  --dashboard-agent-listen-port=52365 \
  --ray-client-server-port=52097

ray start --head --port=45522 \
  --dashboard-port=40926 \
  --dashboard-agent-listen-port=52366 \
  --ray-client-server-port=52098

  3. submit job to first cluster (successful):

    ray job submit --address=http://127.0.0.1:40925/ -- python test.py

Job submission server address: http://127.0.0.1:40925

-------------------------------------------------------
Job 'raysubmit_SNWMzwRLriF5gQJQ' submitted successfully
-------------------------------------------------------

Next steps
  Query the logs of the job:
    ray job logs raysubmit_SNWMzwRLriF5gQJQ
  Query the status of the job:
    ray job status raysubmit_SNWMzwRLriF5gQJQ
  Request the job to be stopped:
    ray job stop raysubmit_SNWMzwRLriF5gQJQ

Tailing logs until the job exits (disable with --no-wait):
2024-06-04 21:25:56,172 INFO job_manager.py:530 -- Runtime env is setting up.
Hello, World!

------------------------------------------
Job 'raysubmit_SNWMzwRLriF5gQJQ' succeeded
------------------------------------------

  4. submit job to second cluster (successful):

    ray job submit --address=http://127.0.0.1:40926/ -- python test.py

Job submission server address: http://127.0.0.1:40926

-------------------------------------------------------
Job 'raysubmit_LJpfSimruNCb6kyb' submitted successfully
-------------------------------------------------------

Next steps
  Query the logs of the job:
    ray job logs raysubmit_LJpfSimruNCb6kyb
  Query the status of the job:
    ray job status raysubmit_LJpfSimruNCb6kyb
  Request the job to be stopped:
    ray job stop raysubmit_LJpfSimruNCb6kyb

Tailing logs until the job exits (disable with --no-wait):
2024-07-10 11:53:19,976	INFO job_manager.py:530 -- Runtime env is setting up.
Hello, World!

------------------------------------------
Job 'raysubmit_LJpfSimruNCb6kyb' succeeded
------------------------------------------