How to use a Stopper based on an evaluation metric?

Ray Libraries (Data, Train, Tune, Serve) Ray Tune
1

Hi, how do I create a stopper that can work on an evaluation metric? I tried the following:

            def stopper(trial_id, result):
                stop = (
                        ((result['timesteps_total'] > 2e5) and (result['evaluation/episode_reward_mean'] < -1.0)) |
                        ((result['timesteps_total'] > 5e5) and (result['evaluation/episode_reward_mean'] < -0.5)) |
                        ((result['timesteps_total'] > 1e6) and (result['evaluation/episode_reward_mean'] < -0.25)) |
                        ((result['timesteps_total'] > 2e6) and (result['evaluation/episode_reward_mean'] < 0))
                )
                return stop

When I then run ray.tune(stop=stopper) I get the following error message:
KeyError: 'evaluation/episode_reward_mean'

My evaluation_interval is set to 1.
I’m using rllib (not sure if this matters)

2

Hey @ihopethiswillfi,

Can you share your code to reproduce this?

It may be that the metric is nested:

-result['evaluation/episode_reward_mean']
+result['evaluation']['episode_reward_mean']
3

Your solution works! Many thanks.

It is however counter-intuitive, because to tune.run() I’m successfully passing as argument: metric="evaluation/episode_reward_mean"