Compatibility of torch API's DataLoader and ray Dataset

1

Hi ,

Developing code based on the examples

https://docs.ray.io/en/master/raysgd/raysgd_pytorch.html

I have to read data from database and load onto ray framework.

For torch.utils.data import DataLoader class, have passed from ray.util.sgd.data.dataset, like below snippet.

 train_dl = DataLoader(train_dataset, batch_size=32, shuffle=True)
 test_dl = DataLoader(test_dataset, batch_size=32, shuffle=False)

It gives the error TypeError: object of type ‘Dataset’ has no len()

Are these two classes compatible , or is there a work around for these?

2

If you are using the Ray SGD Dataset, then you shouldn’t wrap it in a DataLoader yourself. You just need to pass it in directly to TorchTrainer.train