Issue in integrating Ray Serve with Ray's cross-language programming

Ray Libraries (Data, Train, Tune, Serve) Ray Serve
1

I am able to create multiple api’s using ray serve but when I am trying to integrate ray serve with Ray’s cross language programming ( https://docs.ray.io/en/master/cross-language.html ) i am getting issue.

I am using this code

import ray
from ray import serve
import sys
 
sys.path.append(".../Desktop/Cross_language_programming/ADD.jar")
ray.init(job_config=ray.job_config.JobConfig(code_search_path=sys.path))
 
serve.start()
 
@serve.deployment
def add_num(request):
 add_function = ray.java_function("io.ray.demo.Math", "add")
 # Call the Java remote function.
 obj_ref3 = add_function.remote(1, 2)
 res = ray.get(obj_ref3)
 return res
 
 
add_num.deploy()

I am getting this error

2021-06-29 15:59:38,982 INFO services.py:1272 -- View the Ray dashboard at http://127.0.0.1:8266
(pid=62752) 2021-06-29 15:59:41,278     INFO http_state.py:72 -- Starting HTTP proxy with name 'hjNgRg:SERVE_CONTROLLER_ACTOR:SERVE_PROXY_ACTOR-node:192.168.1.9-0' on node 'node:192.168.1.9-0' listening on '127.0.0.1:8000'
(pid=62755) INFO:     Started server process [62755]
2021-06-29 15:59:42,502 INFO api.py:415 -- Updating deployment 'add_num'.
(pid=62752) 2021-06-29 15:59:42,550     INFO backend_state.py:773 -- Adding 1 replicas to backend 'add_num'.
(pid=62752) 2021-06-29 15:59:42,559     INFO runtime_env.py:111 -- Using preinstalled conda environment: base
2021-06-29 15:59:45,444 WARNING worker.py:1114 -- Traceback (most recent call last):
  File "/Users/abhishekjadhav/Desktop/RayServe/env5/lib/python3.8/site-packages/ray/_private/function_manager.py", line 452, in _load_actor_class_from_local
    object = getattr(object, part)
AttributeError: 'function' object has no attribute '<locals>'

 

During handling of the above exception, another exception occurred:

 

Traceback (most recent call last):
  File "python/ray/_raylet.pyx", line 595, in ray._raylet.task_execution_handler
  File "python/ray/_raylet.pyx", line 394, in ray._raylet.execute_task
  File "/Users/abhishekjadhav/Desktop/RayServe/env5/lib/python3.8/site-packages/ray/_private/function_manager.py", line 398, in load_actor_class
    actor_class = self._load_actor_class_from_local(
  File "/Users/abhishekjadhav/Desktop/RayServe/env5/lib/python3.8/site-packages/ray/_private/function_manager.py", line 458, in _load_actor_class_from_local
    raise RuntimeError(
RuntimeError: Actor create_backend_replica.<locals>.RayServeWrappedReplica failed to be imported from local code.
sys.path: ['/Users/abhishekjadhav/Desktop/Cross_language_programming/ADD.jar', '/Users/abhishekjadhav/Desktop/Cross_language_programming', '/Library/Frameworks/Python.framework/Versions/3.8/lib/python38.zip', '/Users/abhishekjadhav/Desktop/RayServe/env5/lib/python3.8/site-packages'],
Error Message: 'function' object has no attribute '<locals>'
An unexpected internal error occurred while the worker was executing a task.
2021-06-29 15:59:45,444 WARNING worker.py:1114 -- A worker died or was killed while executing a task by an unexpected system error. To troubleshoot the problem, check the logs for the dead worker. Task ID: ffffffffffffffffc3d56a25b0b2f47e61da2db201000000 Worker ID: 80961672a52af9c8262d2f5843acd4d7033456d6c3dfe1da83a7831d Node ID: b63eaa795049bb0c3785ca5f7b4d23325c028f683bd4b2017f0ded89 Worker IP address: 192.168.1.9 Worker port: 60283 Worker PID: 62759
(pid=62752) 2021-06-29 15:59:45,471     ERROR controller.py:121 -- Exception updating backend state: Failed to look up actor with name 'hjNgRg:SERVE_CONTROLLER_ACTOR:add_num#bCGNdV'. You are either trying to look up a named actor you didn't create, the named actor died, or the actor hasn't been created because named actor creation is asynchronous.
(pid=62759) 2021-06-29 15:59:45,433     ERROR worker.py:409 -- SystemExit was raised from the worker

Ray version 1.4.0
Python version 3.8.7

1 Like
2

@architkulkarni, any help is much appreciated.

1 Like
3

Hi @Tarang, thanks for raising this issue and for providing a minimal reproduction! I’m assuming this works fine if you remove Ray Serve completely and just use an ordinary remote function, right?

This does seem to be a bug, and I’m not sure off the top of my head what could be causing it. Would you mind creating an issue here? Issues · ray-project/ray · GitHub Then the team can investigate further.

2 Likes
4

@architkulkarni thanks for the quick response .

Yes you are right it works absolutely fine without serve

End result for us is to have ray serve api which can support cross language

1 Like