Object Reference in Ray Queue

How severe does this issue affect your experience of using Ray?

  • High: It blocks me to complete my task.

I’m trying to use ray.util.queue for handling communication between Actors and put a ObjectRef to the queue, but when I try to get the ObjectRef from the queue, I get the actual item back and it’s not mutable. Here is the sample code I have:

import ray
import numpy as np
from ray.util.queue import Queue

q = Queue(1)
arr_ref = ray.put(np.random.rand(90, 1000, 1000))
q.put(arr_ref)
arr_from_queue = q.get()

type(arr_from_queue)
<class 'numpy.ndarray'>

arr_from_queue[0][0][0] = 0
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: assignment destination is read-only

Could somebody help me understand here and suggest what I can do to achieve this if possible?

Hi mark, the arr_from_queue returned is an immutable object shared between Ray processes.
Not an expert on numpy, but does answer in python - NumPy Array Copy-On-Write - Stack Overflow help?

Thanks for raising the question @mark199342 !

Like @Chen_Shen mentioned, Ray object store is based on the design principle of immutable objects. In your example, even without using Queue, the same error would happen.

The immutability design decision is to simplify the consistency model, if interested you can read https://stephanie-wang.github.io/pdfs/nsdi21-ownership.pdf.