Passing configuration

RLlib
1

Hello,

when passing a custom tuner configuration to find optimal hyperparameter for an RL problem, I get the following errer:

WARNING algorithm_config.py:2534 – Setting exploration_config={} because you set _enable_rl_module_api=True. When RLModule API are enabled, exploration_config can not be set. If you want to implement custom exploration behaviour, please modify the forward_exploration method of the RLModule at hand. On configs that have a default exploration config, this must be done with config.exploration_config={}.

I define a standard configuration like:

ppo_config = (
        PPOConfig()
        .environment(
            env=ENVIRONMENT,
            env_config={
                "max_episode_steps": MAX_EPISODE_STEPS,
                "sampling_time": SAMPLING_TIME,
                "render_mode": None,
            },
            render_env=False,
        )
        .rollouts(num_rollout_workers=4, num_envs_per_worker=1)
        .framework("torch")
        .exploration(explore=True)
        )
        .training(
            sgd_minibatch_size=tune.grid_search([8, 16, 32, 64, 128, 256, 512]),
       ))

and configure the tuner with:

    tuner = tune.Tuner(
        "PPO",
        param_space=ppo_config,
        tune_config=tune.TuneConfig(
            metric="episode_reward_mean",
            mode="max",
            num_samples=8,
        ),
        run_config=air.RunConfig(
            stop=stop,
            checkpoint_config=air.CheckpointConfig(
                checkpoint_at_end=True,
                checkpoint_frequency=1,
                num_to_keep=25,
                checkpoint_score_attribute="episode_reward_mean",
                checkpoint_score_order="max",
            ),

        ),
    )
    results = tuner.fit()

Can someone privde help, how to fix this issue?
Thanks

2

Hi @PatrickGoettsch,

The message seems to be a warning and not an error. Are you sure that is the cause of error?

Cc @arturn

3

@PatrickGoettsch The WARNING you posted occurs very frequently. This may seem irritating, but it should not cause an error.

1 Like