[Tune] Feature: possibility to report evaluation metrics

When configuring the CLIReporter for a trial, for instance with

tune.run(..., progress_reporter=CLIReporter(metric_columns=["date", "training_iteration", "timesteps_total"])

it would be nice to be able to report evaluation metrics. The results of the evaluation are nested like this result["evaluation"]["episode_reward_mean"] and this is therefore not possible to access them.

Hmm, I think you can do evaluation/episode_reward_mean to get nested metrics?

Unfortunately, it is not working. I have digged a bit into the code of the CLIReporter and there is only a get call on the results dictionary, therefore it is not possible to report nested metrics.

Maybe is this a feature that is already implemented in some upcoming version ? (I am using ray 1.1.0)

Hmm, I see – could you help post a feature request about this on Github?

No problem, the issue has been opened here [Tune] Feature: possibility to report nested metrics · Issue #13827 · ray-project/ray · GitHub