I am working with a custom simulation environment that occasionally fails and requires a restore to the last checkpoint. However, upon running some experiments with the restore function and digging through the source, it seems as though the memory buffer and episode reward mean are not maintained as part of the restore. We are considering doing callbacks to access these bits of information we need to maintain continuity in our experiment, however, it seems quite difficult to access the memory buffer (perhaps this is just because I am a newbie to Ray/RLlib).
Does anyone have any advice on the best way to retrieve the local_replay_buffer shards from the Ray object store memory for use when restoring the experiment?