Ray for Rapberry Pi, is possible?

Peter_Pirog · January 28, 2021, 6:05pm

Now I tried simple command:

python3 setup.py install

and again the same error:

Alex · January 29, 2021, 3:27am

Hmm I don’t have easy access to a Raspberry Pi (Lost mine ), but can you try this branch? GitHub - wuisawesome/ray at fix_arm

Peter_Pirog · January 29, 2021, 5:03am

@Alex Thank You, I will try it .

Peter_Pirog · January 29, 2021, 5:43pm

@Alex I tested this code by commands with Rapberry pi 4:

git clone --branch fix_arm --recurse-submodules https://github.com/wuisawesome/ray.git
sudo chmod -R 777 ./ray
cd ./ray/python/
python3 setup.py install

The result is:

Peter_Pirog · February 5, 2021, 5:02am

@Alex I used branch https://github.com/wuisawesome/ray/tree/fix_arm

I installed libatomic library and used command with option:

bazel build --copt="-latomic" -c opt //:ray_pkg

Now here is the error:

FAILED: Build did NOT complete successfully
pi@raspberrypi:/tmp/ray $ bazel build --copt="-latomic" -c opt //:ray
INFO: Writing tracer profile to ‘/home/pi/.cache/bazel/_bazel_pi/c81ef09a5cf12e451dde4b8cf7740758/command.profile.gz’
DEBUG: /tmp/ray/bazel/ray_deps_setup.bzl:63:9: No implicit mirrors used because urls were explicitly provided
ERROR: Skipping ‘//:ray’: no such target ‘//:ray’: target ‘ray’ not declared in package ‘’ defined by /tmp/ray/BUILD.bazel
WARNING: Target pattern parsing failed.
ERROR: no such target ‘//:ray’: target ‘ray’ not declared in package ‘’ defined by /tmp/ray/BUILD.bazel

I will be grateful for any suggestions?

Alex · February 5, 2021, 9:20pm

what’s the -c opt for here?

Peter_Pirog · February 6, 2021, 11:26am

@Alex There is no special reason for that, I tested many combinations of options from https://docs.ray.io/en/master/development.html to check if some of its will finish without error.
The option fastbuild:

bazel build --copt="-latomic" -c fastbuild //:ray_pkg

gives error:

unfortunatelly I dont’know how to fix error:

error: undefined reference to ‘__atomic_load_8’

Thank You for the activity in this topic

Alex · February 8, 2021, 12:00am

hmmm i wonder if this is still related to this libatomic stuff. Can you try the ld preload suggestion from here? Undefined reference to __atomic_xxx · Issue #15278 · opencv/opencv · GitHub

Peter_Pirog · February 15, 2021, 1:42pm

@Alex for few days I have built packges in ray requirements files:
https://github.com/PeterPirog/Raspberry_armv7_builds
but now I’m confused which aproach is correct:

try use “python3 setup.py bdist_wheel” command in /ray/python directory
try tu use npn according to Building Ray from Source — Ray v2.0.0.dev0
try tu use Building Ray (Python Only)
try tu use script build.sh from /ray/build.sh
try tu use “bazel build -c fastbuild //:ray_pkg”
try somehow ./configure and make

The goal is still the same build wheel ray file for python 3.8 armv7
when I try option1 “python3 setup.py bdist_wheel” the problem is:

bazel-out/arm-opt/bin/_virtual_includes/ray_common/ray/common/ray_config_def.h:200:75: error: integer overflow in expression of type ‘long int’ results in ‘-2147483648’ [-Werror=overflow]
RAY_CONFIG(uint64_t, object_manager_max_bytes_in_flight, 2L * 1024 * 1024 * 1024)
~~~~~~~~~~~~~^~
bazel-out/arm-opt/bin/virtual_includes/ray_common/ray/common/ray_config.h:35:18: note: in definition of macro ‘RAY_CONFIG’
type name## = default_value;

As I understand it means that long int type is too smal for the variable but how to solve overflow error?

Alex · February 15, 2021, 6:14pm

This is how our CI builds wheels. ray/.travis.yml at master · ray-project/ray · GitHub Eventually this goes down all the paths you mentioned, but you eventually want to be calling option 1.

That warning/error you’re coming across is another example of us assuming an x86 instruction set (64 bit longs). Can you change the 2L to 2ULL and try again?

Peter_Pirog · February 15, 2021, 6:49pm

@Alex Thank You for the answer. I have changed 2L to 2ULL in https://github.com/ray-project/ray/blob/ray-1.2.0/src/ray/common/ray_config_def.h, I will inform about results.

Peter_Pirog · February 15, 2021, 10:18pm

@Alex After changing 2L to 2ULL the errors are:

error: undefined reference to ‘__atomic_load_8’

Alex · February 15, 2021, 10:32pm

Hmm it seems like we’re going in circles. How did you solve this problem? Ray for Rapberry Pi, is possible? - #18 by Peter_Pirog

Peter_Pirog · February 16, 2021, 5:31pm

@Alex when I built opencv_python_headles file I installed:

sudo apt-get install -y libatomic-ops-dev

and set flags:

export CMAKE_ARGS="-DCMAKE_BUILD_TYPE=RELEASE -DCMAKE_INSTALL_PREFIX=~/libraries/opencv -DOPENCV_EXTRA_MODULES_PATH=~/builds/opencv_contrib/modules -DENABLE_NEON=ON -DENABLE_VFPV3=ON -DINSTALL_PYTHON_EXAMPLES=OFF -DCMAKE_CXX_FLAGS=-latomic -DOPENCV_EXTRA_EXE_LINKER_FLAGS=-latomic -DBUILD_NEW_PYTHON_SUPPORT=ON -DBUILD_opencv_python3=ON -DHAVE_opencv_python3=ON -DBUILD_opencv_python2=OFF -DPYTHON_DEFAULT_EXECUTABLE=/home/pi/src/Python-3.8.5/bin/python3.8 -DBUILD_EXAMPLES=OFF -DOPENCV_ENABLE_NONFREE=ON -DWITH_CUDA=OFF"

and it worked for opencv but now I don’t know how to set during compilation path to

/usr/lib/arm-linux-gnueabihf/libatomic.so.1.2.0

I added command below but it still doesn’t work:

export LD_PRELOAD=/usr/lib/arm-linux-gnueabihf/libatomic.so.1.2.0

Peter_Pirog · February 21, 2021, 7:29pm

@Alex I have achieved next phase in raspberry pi ray building. The building process is successful:

Steps:

git clone --recursive --branch ray-1.2.0 https://github.com/ray-project/ray.git
sudo chmod -R 777 ./ray

!!! I have changed 2L to 2ULL in /src/ray/common/ray_config_def.h in line 200 !!!

sudo apt-get install -y libatomic-ops-dev
cd ./ray
export LD_LIBRARY_PATH=/usr/lib/arm-linux-gnueabihf:$LD_LIBRARY_PATH
sudo bazel build --action_env=LD_LIBRARY_PATH --linkopt=-latomic -c opt //:ray_pkg

Now the question is: “How to generate whl file ??” The result is:

Alex · February 22, 2021, 6:35pm

Nice! You can check out how the CI builds wheels here: ray/.travis.yml at master · ray-project/ray · GitHub

you can probably just run python python/setup.py bdist_wheel if everything is set up properly, but in general, the recommendation is to just do as the ci does.

Peter_Pirog · February 22, 2021, 6:49pm

@Alex , Thank you for the suggestions. I tested python setup.py bdist_wheel but the result is my favourite error
error: undefined reference to ‘__atomic_load_8’

maybe in some way I have to set linker variables for it -I’m not sure. Tthe steps to the success are very small but there is still some progress. I will analyze https://github.com/ray-project/ray/blob/master/.travis.yml#L208 and maybe I will find some solution.
Thank You again.

Peter_Pirog · February 23, 2021, 11:05am

@Alex I next step I have corrected:

!!! CHANGE LINE 296 in /src/ray/common/ray_config_def.hray/python/setup.py to
[“build”, “–linkopt=-latomic”, “–”] + bazel_targets, setup.py line 296

and

cd /ray/python
sudo python3 setup.py bdist_wheel
cd /ray/python/dist
python3 -m pip install ray-1.2.0-cp38-cp38-linux_armv7l.whl

whl instalation is correct , next problem to solve is building whl tensorflow==2.4.1 for python 3.8 armv7 (now I have only for 3.7)
Link to WHL file: https://github.com/PeterPirog/Raspberry_armv7_builds/tree/main/ray

Alex · February 23, 2021, 6:21pm

Looks great! Hopefully others who come across this can use this too.

Good luck getting tensorflow built, I doubt we’ll be much help with that though

Peter_Pirog · February 23, 2021, 9:04pm

@Alex Tensorflo build is complete
Here is link for my armv7 tensorflow 2 builds:
https://github.com/PeterPirog/Raspberry_armv7_builds/blob/main/tensorflow/Tensorflow_links.md
Its over 100 MB so I use google drive not github.

and armv7 pytorch builds:
https://github.com/PeterPirog/Raspberry_armv7_builds/tree/main/pytorch

Now I Have to solve next problem why the rllib train is abborted with no errors: