How severe does this issue affect your experience of using Ray?
- None: Just asking a question out of curiosity
- Low: It annoys or frustrates me for a moment.
- Medium: It contributes to significant difficulty to complete my task, but I can work around it.
- High: It blocks me to complete my task.
Hi~ I just make a AI service using Ray cluster
now, I faced with a problem.
It just working well when I use ray cluster (ray start & ray.init)
but, An error occured when I use ray cluster within airflow’s dag (same project, same directory structure)
so, I find some information about runtime_env in ray’s docs, but It was not working uu.
below an error message of airflow
[Note : you can see an word ‘actors’, It just a folder name]
[2022-11-17, 15:42:37 KST] {newsGathering.py:111} INFO - status: 400 errormsg: rootairflowdagsmeerkatutils>processhandler.py status:4042 --> The actor died because of an error raised in its creation task, e[36mray::Crawling.__init__()e[39m (pid=2081179, ip=172.17.0.8, repr=<actors.flows.collect.crawling.FunctionActorManager._create_fake_actor_class.<locals>.TemporaryActor object at 0x7f393a8608b0>)
RuntimeError: The actor with name Crawling failed to import on the worker. This may be because needed library dependencies are not installed in the worker environment:
e[36mray::Crawling.__init__()e[39m (pid=2081179, ip=172.17.0.8, repr=<actors.flows.collect.crawling.FunctionActorManager._create_fake_actor_class.<locals>.TemporaryActor object at 0x7f393a8608b0>)
ModuleNotFoundError: No module named 'actors'
Traceback (most recent call last):
File "/root/airflow/dags/meerkat/utils/processhandler.py", line 76, in parellelRun
rtn = ray.get(funcArr)
File "/usr/local/lib/python3.8/dist-packages/ray/_private/client_mode_hook.py", line 105, in wrapper
return func(*args, **kwargs)
File "/usr/local/lib/python3.8/dist-packages/ray/worker.py", line 1765, in get
raise value
ray.exceptions.RayActorError: The actor died because of an error raised in its creation task, e[36mray::Crawling.__init__()e[39m (pid=2081179, ip=172.17.0.8, repr=<actors.flows.collect.crawling.FunctionActorManager._create_fake_actor_class.<locals>.TemporaryActor object at 0x7f393a8608b0>)
RuntimeError: The actor with name Crawling failed to import on the worker. This may be because needed library dependencies are not installed in the worker environment:
e[36mray::Crawling.__init__()e[39m (pid=2081179, ip=172.17.0.8, repr=<actors.flows.collect.crawling.FunctionActorManager._create_fake_actor_class.<locals>.TemporaryActor object at 0x7f393a8608b0>)
ModuleNotFoundError: No module named 'actors'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/root/airflow/dags/meerkat/tasks/newsGathering.py", line 99, in flow
parellelRun(funcs)
File "/root/airflow/dags/meerkat/utils/processhandler.py", line 82, in parellelRun
raise CommonException(str(e), Status.RAY_PARELLEL_RUN_FAIL.value, getPath(__file__))
base_structures.exceptions.common.CommonException: rootairflowdagsmeerkatutils>processhandler.py status:4042 --> The actor died because of an error raised in its creation task, e[36mray::Crawling.__init__()e[39m (pid=2081179, ip=172.17.0.8, repr=<actors.flows.collect.crawling.FunctionActorManager._create_fake_actor_class.<locals>.TemporaryActor object at 0x7f393a8608b0>)
RuntimeError: The actor with name Crawling failed to import on the worker. This may be because needed library dependencies are not installed in the worker environment:
e[36mray::Crawling.__init__()e[39m (pid=2081179, ip=172.17.0.8, repr=<actors.flows.collect.crawling.FunctionActorManager._create_fake_actor_class.<locals>.TemporaryActor object at 0x7f393a8608b0>)
ModuleNotFoundError: No module named 'actors'
plz, help me~
