How can I deploy my reinforcement learning model trained with tune using the new API?

[Daraan]

I am not entirely sure if its relevant for your case. If not at least its a nice to know, do you know that the episode_return_mean is smoothed by config.metrics_num_episodes_for_smoothing? See the topic I just have posted:

Change the episode metrics as the timeframe is a bit vague?

Currently episode metrics are logged like this with a window in the EnvRunner: win = config.metrics_num_episodes_for_smoothing logger.log_value("return_mean", ret, window=win) logger.log_value("return_min", ret, reduce="min", window=win) logger.log_value("return_max", ret, reduce="max", window=win) Now I don’t think that is the best way to do it, when we do not known the episodes, especially during evaluation this then is off when win != len(episodes_seen). Case 1: win < episo…

In short the min/mean/max you obtain by using local_env_runner.get_metrics() are from the last metrics_num_episodes_for_smoothing sampled episodes - not bound to an iteration.
Furthermore (depending on your ray version), restoring metrics is broken, see [RLlib] Checkpoint metrics loading with Tune is broken in 2.47.0 · Issue #53877 · ray-project/ray · GitHub. In your case I think the smoothing from the old episodes (if your windows reaches there) can be off / lost. So possibly you only get the smoothed value from after you loaded the checkpoint.

Maybe you have to cross check if things are maybe correct but not logged like you would expect.
Cheers, and good luck.