In Ray 2.53, the progress.csv file does not include hist_stats/episode_reward or similar columns by default. To store per-episode rewards (hist_stats), you must set the config option keep_per_episode_custom_metrics = True in your AlgorithmConfig. Without this, only aggregated metrics (like means) are saved, not the full per-episode history (discuss.ray.io: is-hist-stats-no-longer-supported, discuss.ray.io: accessing-custom-metrics-for-episodes).
If you enable keep_per_episode_custom_metrics, the per-episode custom metrics will be available in the results dictionary, but not necessarily in progress.csv. You may need to access them from result.json or directly from the results object in your code (discuss.ray.io: is-hist-stats-no-longer-supported). Would you like more detail on how to access or log these metrics?
Sources:
- discuss.ray.io: is-hist-stats-no-longer-supported
- discuss.ray.io: accessing-custom-metrics-for-episodes
Hint: Mention @RunLLM in the post for followups.