Saving/loading training experience in the D4RL format

How severe does this issue affect your experience of using Ray?

  • High: It blocks me to complete my task.

Hi! I want to record all experience during training of the DQN agent in the D4RL format. How can I do that? Or it there any way to convert standard json outputs to this format?

D4RL saves data in a dict, simply in a one continuous array (or nested array) for each key, like:
data[“obs”] - all observations from first trajectory to the last one after another (without skips). Same for all other keys, like “actions”, “terminated”, “truncated”, “rewards”, etc