We have an IterableDataset object in our application API that uses subclass
torch.utils.data.IterableDataset(). I tried converting that to ray iterator so that the data read can happen in parallel. But it fails with the below error and the link provided also doesn’t exist.
TypeError: Could not serialize the argument <IterableDataset object at 0x7f67c405f070> for a task or actor ray.util.iter.ParallelIteratorWorker.__init__. Check https://docs.ray.io/en/master/serialization.html#troubleshooting for more information.
Any inputs on how the above can be achieved?