"No module named 'ray.tests'" when running Python tests locally

1

Hi team,

I am trying to build and test ray locally using the following commands (assuming dependencies are installed). Getting a number of failures that say:

E ModuleNotFoundError: No module named 'ray.tests'

Commands:

git clone https://github.com/ray-project/ray
cd ray
git checkout 2.23.0
cd python/
pip install -r requirements.txt
pip install -e . --verbose
python setup.py bdist_wheel
pip install dist/ray-2.23-cp311-cp311-linux_x86_64.whl
cd ..
pip install -c python/requirements_compiled.txt -r python/requirements/test-requirements.txt
python -m pytest -v -s python/ray/tests

Please help.

2

What OS are you trying to build it for? Are you following the instructions in Building Ray from Source — Ray 2.30.0?

3

@Sam_Chan I am building it on UBI 9.3 (RHEL). I removed this line and the error was gone:
pip install dist/ray-2.23-cp311-cp311-linux_x86_64.whl

Yes I am following the instructions (except for the venv because I am building in a fresh container). The tests are still not running, it seems to hang.

4 
[root@sumit-osa3-isv tests]# python -m pytest
Test session starts (platform: linux, Python 3.9.19, pytest 7.0.1, pytest-sugar 0.9.5)
rootdir: /sumit/ray, configfile: pytest.ini
plugins: shutil-1.7.0, sphinx-0.5.1.dev0, docker-tools-3.1.3, timeout-2.1.0, sugar-0.9.5, anyio-3.7.1, lazy-fixture-0.6.3, httpserver-1.0.6, asyncio-0.16.0, hypothesis-6.104.2, rerunfailures-11.1.2, forked-1.4.0, virtualenv-1.7.0
timeout: 180.0s
timeout method: signal
timeout func_only: False
collecting ...
―――――――――――――――――――――――――――――――――――――――――――― ERROR collecting python/ray/tests/horovod/test_horovod.py ――――――――――――――――――――――――――――――――――――――――――――
horovod/test_horovod.py:65: in <module>
    @pytest.mark.skipif(not gloo_built(), reason="Gloo is required for Ray integration")
E   NameError: name 'gloo_built' is not defined

―――――――――――――――――――――――――――――――――――――――――――――― ERROR collecting python/ray/tests/modin/test_modin.py ――――――――――――――――――――――――――――――――――――――――――――――
ImportError while importing test module '/sumit/ray/python/ray/tests/modin/test_modin.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/lib64/python3.9/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
modin/test_modin.py:39: in <module>
    from ray.tests.modin.modin_test_utils import df_equals
modin/modin_test_utils.py:21: in <module>
    from modin.utils import to_pandas
E   ImportError: cannot import name 'to_pandas' from 'modin.utils' (/usr/local/lib/python3.9/site-packages/modin/utils.py)

――――――――――――――――――――――――――――――――― ERROR collecting python/ray/tests/runtime_env_container/test_log_file_exists.py ―――――――――――――――――――――――――――――――――
runtime_env_container/test_log_file_exists.py:37: in <module>
    f.remote()
../remote_function.py:139: in _remote_proxy
    return self._remote(args=args, kwargs=kwargs, **self._default_options)
../_private/auto_init_hook.py:21: in auto_init_wrapper
    return fn(*args, **kwargs)
../util/tracing/tracing_helper.py:310: in _invocation_remote_span
    return method(self, args, kwargs, *_args, **_kwargs)
../remote_function.py:332: in _remote
    runtime_env = parse_runtime_env(task_options["runtime_env"])
../_private/utils.py:1670: in parse_runtime_env
    return RuntimeEnv(**(runtime_env or {}))
../runtime_env/runtime_env.py:365: in __init__
    raise ValueError(
E   ValueError: The 'container' field currently cannot be used together with other fields of runtime_env. Specified fields: dict_keys(['_inject_current_ray', 'container'])
----------------------------------------------------------------- Captured stderr -----------------------------------------------------------------
2024-07-02 12:19:02,543 INFO worker.py:1740 -- Started a local Ray instance. View the dashboard at 127.0.0.1:8265
[2024-07-02 12:19:02,546 I 64802 64802] logging.cc:234: Set ray log level from environment variable RAY_BACKEND_LOG_LEVEL to 0
(raylet) [2024-07-02 12:19:02,505 I 64999 64999] logging.cc:234: Set ray log level from environment variable RAY_BACKEND_LOG_LEVEL to 0

――――――――――――――――――――――――――――――――――――― ERROR collecting python/ray/tests/runtime_env_container/test_put_get.py ―――――――――――――――――――――――――――――――――――――
runtime_env_container/test_put_get.py:22: in <module>
    wrapped_ref = create_ref.remote()
../remote_function.py:139: in _remote_proxy
    return self._remote(args=args, kwargs=kwargs, **self._default_options)
../_private/auto_init_hook.py:21: in auto_init_wrapper
    return fn(*args, **kwargs)
../util/tracing/tracing_helper.py:310: in _invocation_remote_span
    return method(self, args, kwargs, *_args, **_kwargs)
../remote_function.py:332: in _remote
    runtime_env = parse_runtime_env(task_options["runtime_env"])
../_private/utils.py:1670: in parse_runtime_env
    return RuntimeEnv(**(runtime_env or {}))
../runtime_env/runtime_env.py:365: in __init__
    raise ValueError(
E   ValueError: The 'container' field currently cannot be used together with other fields of runtime_env. Specified fields: dict_keys(['_inject_current_ray', 'container'])

―――――――――――――――――――――――――――――――――― ERROR collecting python/ray/tests/runtime_env_container/test_ray_env_vars.py ―――――――――――――――――――――――――――――――――――
runtime_env_container/test_ray_env_vars.py:24: in <module>
    assert ray.get(f.remote()) == "1"
../remote_function.py:139: in _remote_proxy
    return self._remote(args=args, kwargs=kwargs, **self._default_options)
../_private/auto_init_hook.py:21: in auto_init_wrapper
    return fn(*args, **kwargs)
../util/tracing/tracing_helper.py:310: in _invocation_remote_span
    return method(self, args, kwargs, *_args, **_kwargs)
../remote_function.py:332: in _remote
    runtime_env = parse_runtime_env(task_options["runtime_env"])
../_private/utils.py:1670: in parse_runtime_env
    return RuntimeEnv(**(runtime_env or {}))
../runtime_env/runtime_env.py:365: in __init__
    raise ValueError(
E   ValueError: The 'container' field currently cannot be used together with other fields of runtime_env. Specified fields: dict_keys(['_inject_current_ray', 'container'])
5

when developing, after you pip install -e you do not need to build and install the wheel anymore. -e means symbolic link the directory, which will make ray.tests importable.

6

@Lonnie_Liu yes, that issue has resolved. Please check above for logs on a new one.

7

you need to install horovod to run that test.

8

ray has many different tests, and they require different environment set up. you should not try to run all the test together and expect them to all pass.

9

@Lonnie_Liu thanks, the tests do execute when run individually. I missed that in the documentation.
My ultimate goal is to port ray to IBM power (ppc64le) and open a PR. Can you tell me which of the Python tests would be relevant to validate the port? There are tests that are passing on x86 but failing on Power. The C++ unit tests seem to pass with 3 of them failing both on Power and x86.