How severe does this issue affect your experience of using Ray?
- High: It blocks me to complete my task.
Since upgrading to Ray 2.2.0 the custom_eval_function appears to lose access to evaluation_config. See an example revised on top of custom_eval.py;
def custom_eval_function(algorithm, eval_workers):
"""Example of a custom evaluation function.
Args:
algorithm: Algorithm class to evaluate.
eval_workers: Evaluation WorkerSet.
Returns:
metrics: Evaluation metrics dict.
"""
worker = eval_workers.remote_workers()[0] # type(worker) is ray.actor.ActorHandle
this_config = ray.get(worker.creation_args.remote())
print(this_config)
which shows this_config as a dict:
{‘env_creator’: <function at 0x7f2c87ac6310>, ‘validate_env’: None, ‘tf_session_creator’: <function WorkerSet._make_worker..session_creator at 0x7f29ab82b700>, ‘config’: <ray.rllib.algorithms.a2c.a2c.A2CConfig object at 0x7f2ca5f8d6d0>, ‘worker_index’: 1, ‘num_workers’: 1, ‘recreated_worker’: False, ‘log_dir’: ‘/home/ec2-user/SageMaker/ray_results/A2C/A2C_RokuSession-v1_7dbad_00000_0_2023-01-10_06-03-23/’, ‘spaces’: None, ‘default_policy_class’: <class ‘ray.rllib.algorithms.a3c.a3c_torch_policy.A3CTorchPolicy’>, ‘dataset_shards’: None, ‘policy_config’: -1, ‘input_creator’: -1, ‘output_creator’: -1, ‘rollout_fragment_length’: -1, ‘count_steps_by’: -1, ‘batch_mode’: -1, ‘episode_horizon’: -1, ‘preprocessor_pref’: -1, ‘sample_async’: -1, ‘compress_observations’: -1, ‘num_envs’: -1, ‘observation_fn’: -1, ‘clip_rewards’: -1, ‘normalize_actions’: -1, ‘clip_actions’: -1, ‘env_config’: -1, ‘model_config’: -1, ‘remote_worker_envs’: -1, ‘remote_env_batch_wait_ms’: -1, ‘soft_horizon’: -1, ‘no_done_at_end’: -1, ‘fake_sampler’: -1, ‘seed’: -1, ‘log_level’: -1, ‘callbacks’: -1, ‘disable_env_checking’: -1, ‘policy_spec’: -1, ‘policy_mapping_fn’: -1, ‘policies_to_train’: -1, ‘extra_python_environs’: -1, ‘policy’: -1}
However, it seems most of the rllib configs were lost. When I was running in Ray 2.1.0, I was able to read this_config['policy_config']['evaluation_config']['env_config'] but now there’s nothing. Is there a way to retrieve the evaluation_config so that I can read from custom_eval_function()? Using this_config['config'].evaluation_config returns None in Ray 2.2.0.