Corrupted local dev environment after `setup-dev.py`

How severe does this issue affect your experience of using Ray?

  • Low: I found a workaround, but I’d like to understand the root cause if possible and prevent other folks to face the same issue.

I’m setting up my local development environment according to building-ray-python-only instructions. More specifically, I’m doing:

conda create -c conda-forge python=3.9 -n ray-dev
conda activate ray-dev
git clone git@github.com:ray-project/ray.git
cd ray
pip install -U ray-3.0.0.dev0-cp39-cp39-macosx_10_15_x86_64.whl
pip install -c python/requirements_compiled.txt \
  -r python/requirements/test-requirements.txt
python python/ray/setup-dev.py

When running the basic tests using python -m pytest -v -s python/ray/tests/test_basic.py
I get the following error:

2024-01-21 09:50:57,180	INFO worker.py:1762 -- \
Started a local Ray instance. \

[2024-01-21 09:50:57,716 E 19531 6034397] core_worker.cc:215:  \
Failed to register worker \
01000000ffffffffffffffffffffffffffffffffffffffffffffffff to Raylet. \
IOError: \
[RayletClient] Unable to register worker with raylet. \
No such file or directory

This also happens with a simple hello world ray script, so the installation under ‘ray-dev’ is corrupted after running python python/ray/setup-dev.py.

I played with it and noticed and this error only happens if the link from miniconda3/envs/ray-dev/lib/python3.9/site-packages/ray/_private to ray/python/ray/_private is established using the setup-dev.py.

Has anybody seen this error ? why should I or should I not establish that link for the _private folder. By the way there are many differences between the installed and the repo folders.

For now, I’m just skipping this folder using python python/ray/setup-dev.py -y --skip _private, but wonder what’s the root cause and if this could affect other folks as well.

Thanks in advance !