Formatting output file names & organizing saved episodes

TLDR: How do I format the file name of evaluation output files to include the current training step? Target format for single episodes similar to: output-{TRAINSTEP}_worker{ID}_{BATCH}.json

Full story:
I’m rather new to the ray architecture and I’m surprised how easy it is to execute complex RL models. Thanks to everybody who is contributing! Still I am having trouble finding my way through the architecture.

I am training a Multi-Agent model using PPO in a custom environment. To assess the performance of my model during training time, I want to evaluate my model periodically. Fortunately, I can parameterize my Trainer as follows to save my evaluation episodes:

 "evaluation_interval": int(5),
 "evaluation_num_episodes": 1,
 "evaluation_config": {
     "explore": False,
     "output_compress_columns": [],
     "output": "eval_results/",

Now, every 5 iterations, the evaluation worker will gather a single episode and dump it into a json file, which gets split into parts if a size threshold is reached.

Since I would like to use these evaluation episodes to judge how the training process is progressing, I would like to match the episode to the training step. Until now, I am not sure how to retrieve the training step from the data inside the json file. Furthermore, I think it would be more elegant to split the episode files completely: Each worker gather its trajectories and creates files similar to the previous structure, maybe following a filename format like output-{TRAINSTEP}-{WALLTIME}_worker{ID}_{BATCH}.json. I assume the splitting can be achieved by setting "output_max_file_size": 0. The renaming, however, remains a mystery to me.

I hope there is someone who can help me with this. Thanks!

Hey Andre. Thanks for raising this issue. I don’t think formatting the file names is currently possible (could you file a github issue, requesting this as a new feature and assign me (sven1977) to this?).
On the splitting by episode: Did you actually try setting the max_file_size to 0 and confirm this would work?

1 Like

Hey Sven! Thanks, I will file a github issue. I can confirm that setting max_file_size=0 does create new files for each recorded episode.

Link to github issue

Responded on the issue. Will do a PR on this and ping you.

1 Like