Send customised metric from gym to ASyncHyperBand in Tune

RLlib Ray Tune stopping condition & comparisons
1

I am working on a customised gym environment under RLLIB, and the step function of the wrapper returns the following:

return obs, reward, done, {"cus_metric": self.some_customised_metric}

After reading this (Training (tune.Trainable, session.report) — Ray 2.2.0) and this (Training (tune.Trainable, session.report) — Ray 2.2.0) I was under the impression that the following code will pick up the customised metric:

tuner = Tune.Tuner("SAC", tune_config = TuneConfig(
        scheduler = ASHAScheduler(metric = "cus_metric", mode = "max"),
)

But I keep getting the error message:

ValueError: Trial returned a result which did not include the specified metric(s) "acc_value" that "AsyncHyperBandScheduler" expects. Make sure your calls to tune.report() include the metric, or set the TUNE_DISABLE_STRICT_METRIC_CHECKING environment variable to 1.

Could someone please shed light on this and show me how I can pass the customized metric to ASHAScheduler? Thanks a lot!

2

This metric is not returned from your RLlib Algorithm to tune.
You can use an air run config like the follwing…
run_config=air.RunConfig(..., verbose=2)
… to see which values are actually returned to tune from the tune Trainable - in this case an RLlib Algorithm. Tune has no notion of RL concepts like an environment. It only reacts to metrics that RLlib/you report back.

→ So you need to make sure that the metric you are referring to is aggregated from the batches that RLlib collects and also logged as a custom metric to tune. You ca do this with the callback on_learn_on_batch().

Let us know if this works for you :slight_smile: