Ray example - custom evaluation function fails

RLlib
1

Hi,

After upgrading to Ray 2.36.0, my custom evaluation function is failing. To troubleshoot, I tested the custom evaluation function from the documentation, but it also fails with the (same) following error:

TypeError: custom_eval_function() missing 2 required positional arguments: 'algorithm' and 'eval_workers'

(the error originates from ray/rllib/algorithms/algorithm.py:1082)

Does this imply that the custom evaluation function can not take any arguments in ray 2.36? Is there an updated example that I could use for inspiration?

Below is a script to reproduce the issue:

from ray.rllib.examples.evaluation.custom_evaluation import custom_eval_function
from ray.tune.registry import get_trainable_cls
from ray.rllib.examples.envs.classes.simple_corridor import SimpleCorridor
from ray.rllib.algorithms.algorithm_config import AlgorithmConfig

evaluation_parallel_to_training = True 
base_config = (
        get_trainable_cls("PPO")
        .get_default_config()
        .environment(
            SimpleCorridor,
            env_config={"corridor_length": 5},
        )
        .framework(framework='tf2')
        .evaluation(
            custom_evaluation_function=custom_eval_function,
            evaluation_num_env_runners=2,
            evaluation_interval=1,
            evaluation_duration="auto" if evaluation_parallel_to_training else 10,
            evaluation_parallel_to_training=evaluation_parallel_to_training,
            evaluation_config=AlgorithmConfig.overrides(
                env_config={"corridor_length": 7},
                metrics_num_episodes_for_smoothing=5,
                evaluation_duration_unit='timesteps'
            ),
        )
    )
algo = base_config.build()

for _ in range(10):
        result = algo.train()

Environment details:
python 3.11
ray 2.36.0
tensorflow 2.15.1