Pipeline with queues between the actors

In my application I’m using the actors that are running in parallel and there is a queue (ray.util.queue.Queue) that is shared between each two linked actors.

E.g. one actor is reading some file or waiting for a camera input, while the other is processing the previous frame and another one saving file to disk.
source → queue ← worker_1 → queue ← worker_2

To not serialize the big files between the actors I was trying to implement a data actor that has set and get methods to hold the data that cannot be serialized for the queue.

Is there a pattern for that already created? I don’t want to re-invent the wheel. I feel like the pipeline system is quite similar, however it’s not running in parallel.

Hi @siadajpan

Isn’t ray.util.queue.Queue the data actor you want?