How to use Ray with Django environment?

1

I’m trying to use ray with Django environment, I have a script that uses Django’s models to save and retrieve data, for that I’m setting the Django environment and calling django.setup() function to setup the environment, but when I’m using ray the environment is not recognized.

import ray
ray.init()

sys.path.append(script_path)

os.environ['DJANGO_SETTINGS_MODULE']='django_app.settings'

django.setup()
from django_app import models

@ray.remote
def get_data():
    return models.MyModel.objects.all()

error:
(pid=25432) import(name)
(pid=25432) ModuleNotFoundError: No module named ‘django_app’
Also this error:

File "python/ray/_raylet.pyx", line 422, in ray._raylet.execute_task
File "python/ray/_raylet.pyx", line 425, in ray._raylet.execute_task
File "python/ray/_raylet.pyx", line 442, in ray._raylet.execute_task
lib/python3.6/site-packages/ray/serialization.py", line 310, in deserialize_objects
self._deserialize_object(data, metadata, object_ref))
File "/home/arham/observenv/lib/python3.6/site-packages/ray/serialization.py", line
226, in _deserialize_msgpack_data
python_objects = self._deserialize_pickle5_data(pickle5_data)
/lib/python3.6/site-packages/ray/serialization.py", line 216, in _deserialize_pickle5_data
obj = pickle.loads(in_band)
File "/home/arham/observenv/lib/python3.6/site-packages/django/db/models/base.py",
line 1975, in model_unpickle
model = apps.get_model(*model_id)
/lib/python3.6/site-packages/django/apps/registry.py", line 198, in get_model
self.check_models_ready()
line 140, in check_models_ready
raise AppRegistryNotReady("Models aren't loaded yet.")
django.core.exceptions.AppRegistryNotReady: Models aren't loaded yet.
2

Note that the remote task is running in a separate process. That says, the process won’t have django setup there.

I think if you run django.setup() inside a task, it might be resolvable, but I am not sure if that’s the best way to do this.

1 Like
3

Yes @sangcho you are right, each task runs independently with its own environment, so to solve this I created a class with annotation @ray.remote and calling django.setup() inside the constructor. Here is the example code:

class Controller(object):
    def __init__(self):
        os.environ['DJANGO_SETTINGS_MODULE']='django_app.settings'
        django.setup()
    def helper_methods(self):
        pass
4

@_groundbreaker can you perhaps give a more fully fleshed out example? I’m still having a hard time getting this to work with Django. Thanks!

5

Did you solve the problem? I think I meet the same one

6

you can init ray in the settings.py first
try it