Compiling C++ files, packaging in container and deployment


Can you please help share pointers on how do users compile C++ code changes present in the workspace, and package code changes in the container for deployments in Kubernetes?

CC: @Dmitri


@sangcho @ijrsvt
In other words, how should one compile local Ray core C++ changes and build the changes into a Docker image.

I think the way you can do this is to actually clone ray, build it locally Building Ray from Source — Ray v1.4.1, and package them as a container.

ok, thanks @sangcho , I will try those steps, can you please share sample Dockerfile that I can use to add C++ code changes to the container?

I’d probably just push the change to the github or other source control systems and checkout to that branch. Is what you are looking for re-building the wheel after the container is created?

I imagine you could build the wheel locally or wherever you’d like and have a Dockerfile that copies the wheel into the container and pip installs there.

Hello,I followed the documentation, I get the below error:

Exception information:
Traceback (most recent call last):
  File "/home/asmalvan/anaconda3/envs/ray/lib/python3.7/site-packages/pip/_internal/cli/", line 180, in _main
    status =, args)
  File "/home/asmalvan/anaconda3/envs/ray/lib/python3.7/site-packages/pip/_internal/cli/", line 204, in wrapper
    return func(self, options, args)
  File "/home/asmalvan/anaconda3/envs/ray/lib/python3.7/site-packages/pip/_internal/commands/", line 402, in run
  File "/home/asmalvan/anaconda3/envs/ray/lib/python3.7/site-packages/pip/_internal/req/", line 85, in install_given_reqs
  File "/home/asmalvan/anaconda3/envs/ray/lib/python3.7/site-packages/pip/_internal/req/", line 787, in install
  File "/home/asmalvan/anaconda3/envs/ray/lib/python3.7/site-packages/pip/_internal/operations/install/", line 46, in install_editable
  File "/home/asmalvan/anaconda3/envs/ray/lib/python3.7/site-packages/pip/_internal/utils/", line 244, in call_subprocess
    raise InstallationSubprocessError(proc.returncode, command_desc)
pip._internal.exceptions.InstallationSubprocessError: Command errored out with exit status 1: /home/asmalvan/anaconda3/envs/ray/bin/python -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/home/asmalvan/code/ray141/ray/python/'"'"'; __file__='"'"'/home/asmalvan/code/ray141/ray/python/'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code ='"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' develop --no-deps Check the logs for full command output.
Removed build tracker: '/tmp/pip-req-tracker-u9n0vfp1'```

I am using RHEL8.4 to build

I have used ray base image to build entire ray as mentioned in documentation, I now have a build which is installed in the container:

Installed /ray/python
/home/ray/anaconda3/lib/python3.7/site-packages/setuptools/ UserWarning: Module Cython was already imported from /home/ray/anaconda3/lib/python3.7/site-packages/Cython/, but /ray/python/.eggs/Cython-0.29.23-py3.7-linux-x86_64.egg is being added to sys.path
  pkg_resources.working_set.add(dist, replace=True)
/home/ray/anaconda3/lib/python3.7/site-packages/setuptools/ UserWarning: Module cython was already imported from /home/ray/anaconda3/lib/python3.7/site-packages/, but /ray/python/.eggs/Cython-0.29.23-py3.7-linux-x86_64.egg is being added to sys.path
  pkg_resources.working_set.add(dist, replace=True)

Successfully installed ray-2.0.0.dev0

Will try to commit this container and launch it on minikube cluster as next steps


I managed to compile the entire ray source but still unable to see the changes deployed on the container.

  • I made few logger changes to file
  • Using instructions here I managed to compile Building Ray from Source — Ray v1.4.1 .cc file.
  • I made a wheel by running python bdist_wheel
  • When the created wheel is installed locally via pip install wheel-name.whl shows logger changes for in log file gcs_server.out
  • To deploy logger changes on the minkube, I built a new container and copied created wheel in the container.
  • The wheel is installed inside the container to be deployed by another pip install.
  • The new container when deployed onminikube does not show the logger changes that were made before building the container.

Can you please help, what I am missing here? happy to share more information

Before installing the wheel, make sure Ray is not installed in the container. That’s all I can think of at the moment.

1 Like