Utilizing Ray Dataset for faster loading/saving

How severe does this issue affect your experience of using Ray?

  • Medium: It contributes to significant difficulty to complete my task, but I can work around it.

I had a few questions about the integration of Ray Datasets with RLLib. The offline documentation seems to suggest that there is experimental support for utilizing Ray Datasets.

I have been unable to get this to work when using the same config setup as given in the docs. Do I need a certain version of RLLib installed for Ray Datasets to work (note I do have Ray Datasets installed as well)?

Additionally, by utilizing Ray Datasets is it reasonable to expect an increase in saving/loading speeds if multi-processes are utilized? Currently, Im finding it too time consuming to save data for offline usage as it drastically slows down games when using batch_mode = truncate_episodes and takes far to long when using batch_mode = complete_episodes.

@bdpooles what was the error you got?