Inegration of Ray C++ with Rust

Ray Core
1

Hi,

I have been trying to integrate Ray C++ with a Rust based AI platform.
But so far it appears that it doesn’t work. We are stuck in remote function execution.
E.g. the Ray C++ example is built with the following bazel rules:

cc_library(
  name = "libcray.so",
  srcs = glob([
    "cray.cc",
  ]),
  linkopts = ["-shared"],
  linkstatic = True,
  #linkshared = True,
  deps = [
    ":ray_api",
  ],
)

cc_library(
  name = "cray",
  srcs = [
    "libcray.so",
  ],
  linkstatic = True,
  hdrs = glob([
  ]),
  linkopts = ["-Wl,-rpath,./"],
  strip_include_prefix = "",
  visibility = ["//visibility:public"],
)

cc_binary(
  name = "test1",
  srcs = glob([
    "test1.cc",
  ]),
  data = [
    "test1.so",
  ],
  linkstatic = True,
  deps = [
      ":ray_api", ":cray",
  ],
)

cc_binary(
  name = "test1.so",
  srcs = glob([
    "test1.cc",
  ]),
  linkopts = ["-shared"],
  linkstatic = True,
  deps = [
      ":ray_api", ":cray",
  ],
)

cc_library(
  name = "ray_api",
  srcs = [
    "thirdparty/lib/libray_api.so",
  ],
  hdrs = glob([
    "thirdparty/include/**/*.h",
    "thirdparty/include/**/*.hpp",
  ]),
  linkopts = ["-Wl,-rpath,./"],
  strip_include_prefix = "thirdparty/include",
  visibility = ["//visibility:public"],
)

And the bazel rule I tried for Rust binary is:

load("@rules_rust//rust:defs.bzl", "rust_binary", "rust_shared_library")

rust_shared_library(
  name = "librs_test2",

  # Specifies the source file for the binary.
  srcs = ["rs_test2/src/main.rs"],

  # Specifies dependencies for the binary.
  deps = [
    # Depend on the `substring_library` target, which is the crate you created.
    ":ray_api", ":cray",
  ],
  rustc_flags = ["-C", "link-args=-fPIC", "-C", "link-args=-Wl,-Bstatic", "-C", "link-args=-shared"],
  visibility = ["//visibility:public"],
)

# Declares a Rust binary target with the given name.
rust_binary(
  name = "rs_test2",

  # Specifies the source file for the binary.
  srcs = ["rs_test2/src/main.rs"],
  data= [
    "liblibrs_test2.so",
  ],

  # Specifies dependencies for the binary.
  deps = [
    ":ray_api", ":cray",
  ],

  rustc_flags = ["-C", "link-args=-fPIC", "-C", "link-args=-Wl,-Bstatic"],
)

And we are still stuck with the following errors for rust binary:

[2024-08-20 19:45:50,390 I 55265 55265] function_helper.cc:28: Start loading the library "/home/manojd/.cache/bazel/_bazel_manojd/4fca974a36935e11457502024ffdf8bf/execroot/_main/bazel-out/k8-fastbuild/bin/liblibrs_test2.so".
[2024-08-20 19:45:50,406 I 55265 55265] function_helper.cc:72: The library "/home/manojd/.cache/bazel/_bazel_manojd/4fca974a36935e11457502024ffdf8bf/execroot/_main/bazel-out/k8-fastbuild/bin/liblibrs_test2.so" isn't integrated with Ray, skip it.
[2024-08-20 19:45:50,414 E 55265 55265] task_executor.cc:111: An exception was thrown while executing function(bos_add_nums): ray::internal::RayFunctionNotFound: Executable function not found, the function name bos_add_nums

Though the remote function bos_add_nums() is there in the rust library & binary both:

(.env) manojd@ubuntusrv44:/data/chiral/test/ray-template/bazel-bin$ nm liblibrs_test2.so |grep bos_add_nums
0000000000007130 T bos_add_nums
(.env) manojd@ubuntusrv44:/data/chiral/test/ray-template/bazel-bin$ nm rs_test2 |grep bos_add_nums
0000000000019800 T bos_add_nums

That means Ray runtime is not able to recognizance the Rust symbols at all.
Is there any possibility of having break through on this?

Best regards,

Manoj

  • High: It blocks me to complete my task.
2

Have you declare this function by the macro definition “RAY_REMOTE”?

3

Not by the macro RAY_REMOTE but by the function ray::internal::FunctionManager::Instance().RegisterRemoteFunction()

4

Can you use RAY_REMOTE instead of RegisterRemoteFunction? RegisterRemoteFunction is a internal function which is not exposed to users.

5

Hi @GuyangSong,

Thanks for responding.

But my project requirement is to register a remote function dynamically. So if I can use RegisterRemoteFunction() rather than RAY_REMOTE() then it will be great.

Please let me know if anything can be done to get this work (including code changes in C++ libray_api).

6

You can try to debug the module cpp/include/ray/api/function_manager.h. CPP workers will try to find remote function from the ‘map_invokers_’ of FunctionManager. If you can ensure the functions is registered into this map before invoking, i think this could work.

7

Thanks @GuyangSong for the suggestion.
Will try to figure this out.