[RLlib] Import error when trying to build the docs

RLlib
1

Hi, I’m having trouble setting up my dev environment for ray. That is, building ray for Python dev worked as described here: https://docs.ray.io/en/master/development.html#building-ray-python-only

But building the docs does not work and always fails with an import error when running make html:

Warning, treated as error:
autodoc: failed to import function 'rllib.utils.annotations.PublicAPI' from module 'ray'; the following exception was raised:
No module named 'ray.rllib'

I have ray installed inside a virtualenv on WSL2 (Ubuntu 20, Python 3.8) and inside the Python shell from ray.rllib.utils.annotations import PublicAPI works without problems.

Any idea what’s the issue here and how to resolve it? Just in case I make changes to the docs, I’d like to inspect them locally first.
My guess is that it’s some issue with the make command not using my version ray installed inside my virtualenv, even though, I’m executing make after activating the virtualenv.

2

I usually use a separate environment for building docs.

I create a new virtual env, and then run those pip install commands for doc requirements. I do not install ray actually for that environment. Maybe you could try that?

3

That makes sense and I tried it just now: Created a fresh virtualenv just for building the docs.
I followed the setup guide here: Building Ray from Source — Ray v2.0.0.dev0
So, I ran

cd doc
pip install -r requirements-doc.txt
pip install -U -r requirements-rtd.txt # important for reproducing the deployment environment
make html

The installation went fine but make html breaks with the same error as before:

Warning, treated as error:
autodoc: failed to import function 'rllib.utils.annotations.PublicAPI' from module 'ray'; the following exception was raised:
No module named 'ray.rllib'
make: *** [Makefile:57: html] Error 2

Do I need to deviate from the described process somehow? Seems like make html does require ray and doesn’t find it for some reason.

Any ideas?

4

Hey @stefanbschneider , the above works all fine on my Mac with py3.8:

conda create -n ray_docs python=3.8
conda activate ray_docs
cd doc
pip install -r requirements-doc.txt
pip install -U -r requirements-rtd.txt
make html

I do get an error, though, when I try: python; from ray import rllib, which makes sense.
Just out of curiosity, could you try doing this pip install ray[rllib] before building the docs?

5

Hm. I installed ray[rllib] and tried again (python; from ray import rllib works from within my virtualenv), but I still get the same error (here in full):

(venv-docs) stefan@.../ray/doc$ make html
sphinx-build -W -b html -d _build/doctrees   source _build/html
Running Sphinx v3.0.4
generating gallery...
Using Sphinx-Gallery to convert rst text blocks to markdown for .ipynb files.
generating gallery for auto_examples... [100%] plot_pong_example.py
generating gallery for tune/tutorials... [100%] tune-serve-integration-mnist.py
building [mo]: targets for 0 po files that are out of date
building [html]: targets for 192 source files that are out of date
updating environment: [new config] 192 added, 0 changed, 0 removed
reading sources... [100%] xgboost-ray

Warning, treated as error:
autodoc: failed to import function 'rllib.utils.annotations.PublicAPI' from module 'ray'; the following exception was raised:
No module named 'ray.rllib'
make: *** [Makefile:57: html] Error 2

To me it seems like Sphinx is somehow not using my ray[rllib] installation from within the activated virtualenv (made with virtualenv --python=python3 venv-docs).

For now, it’s not too important and not blocking at all since I don’t have any doc changes planned yet. I can check this again in more detail, once I really need it.

6

I just created a new, clean virtualenv for building the docs. In the empty virtualenv, I installed the dependencies as described here:

cd doc
pip install -r requirements-doc.txt
pip install -U -r requirements-rtd.txt # important for reproducing the deployment environment
make html

The install works fine but make html fails as before: autodoc: failed to import function 'rllib.utils.annotations.PublicAPI' from module 'ray'; the following exception was raised: No module named 'ray.rllib'

This seems to be related: python - sphinx-build fail - autodoc can't import/find module - Stack Overflow

I did not explicitly install ray but, I guess it was installed as part of the requirements (eg, xgboost-ray):

$ pip show ray
Name: ray
Version: 1.5.0
Summary: Ray provides a simple, universal API for building distributed applications.
Home-page: https://github.com/ray-project/ray
Author: Ray Team
Author-email: ray-dev@googlegroups.com
License: Apache 2.0
Location: /mnt/c/Users/Stefan/git-repos/work/ray/venv-docs/lib/python3.8/site-packages
Requires: click, pydantic, jsonschema, aiohttp-cors, opencensus, protobuf, gpustat, requests, filelock, redis, aioredis, pyyaml, grpcio, numpy, aiohttp, py-spy, colorama, prometheus-client, msgpack
Required-by: xgboost-ray, tune-sklearn

It’s missing the RLlib dependencies though, eg, gym.

With pip install -U ray[rllib] these dependencies are installed, but the error remains the same.
But running python in my virtualenv, I can run from ray.rllib.utils.annotations import PublicAPI without errors…

The error does not really make sense to me, since sys.path does point to the correct Python installation inside the virtualenv and there, ray is installed and there is a module ray.rllib in site-packages.