[Core] Ray with Nuitka

Ray Core
1

Hi, I’ve been asked to obfuscate our code, so I was trying to use nuitka for it. The problem is that I receive an error:

SustemError: unknown opcode

which apparently comes from: client_mode_hook:
raise value.as_instanceof_cause()

Now all I do is:

        @ray.remote(num_gpus=os.getenv('NUM_GPUS', 0))
        class Wrk:
            def __init__(self):
             .........
             some code
             ..........
            def run(self):
             some code

        def main():
            ray.init(_metrics_export_port=9090, dashboard_host="0.0.0.0", dashboard_port=8260)

        worker = Wrk.remote()

        ray.get(worker.run.remote())



and the stack trace tells me that:

Traceback (most recent call last):
File "/home/delio/PycharmProjects/taweret/cmd/taweret.py", line 51, in <module>
 main()
File "/home/delio/PycharmProjects/taweret/cmd/taweret.py", line 48, in main
 ray.get(worker.run.remote())
File "/home/delio/anaconda3/envs/taweret/lib/python3.7/site-packages/ray/_private/client_mode_hook.py", line 47, in wrapper
 return func(*args, **kwargs)
File "/home/delio/anaconda3/envs/taweret/lib/python3.7/site-packages/ray/worker.py", line 1456, in get
 raise value.as_instanceof_cause()
ray.exceptions.RayTaskError(SystemError): ray::Wrk.run() (pid=32761, ip=192.168.0.160)
File "python/ray/_raylet.pyx", line 439, in ray._raylet.execute_task
File "python/ray/_raylet.pyx", line 473, in ray._raylet.execute_task
File "python/ray/_raylet.pyx", line 476, in ray._raylet.execute_task
File "python/ray/_raylet.pyx", line 480, in ray._raylet.execute_task
File "python/ray/_raylet.pyx", line 432, in ray._raylet.execute_task.function_executor
File "/home/delio/PycharmProjects/taweret/cmd/taweret.py", line 19, in __init__
 def __init__(self):
SystemError: unknown opcode
1 Like
2

Sorry, I’m not very familiar with Nuitka, so correct me if I was wrong.

Right now, ray use pickle to transfer functions of the main script. But from what I read in Nuitka, it looks like it’ll do some complication. I’m think whether it’s because of these compiled functions can’t be pickled and unpicked to run on a remote node.

1 Like