# ValueError: \`RLModule(config=\[RLModuleConfig\])\` has been deprecated
**URL:** https://discuss.ray.io/t/valueerror-rlmodule-config-rlmoduleconfig-has-been-deprecated/21755
**Category:** RLlib
**Created:** [February 11, 2025, 3:11pm UTC](https://discuss.ray.io/t/valueerror-rlmodule-config-rlmoduleconfig-has-been-deprecated/21755 "2025-02-11T15:11:35Z")
**Posts on this page:** 6
**Page:** 1
### Author:  [@zr2358](https://discuss.ray.io/u/zr2358)
#### Post date: [February 11, 2025, 3:11pm UTC](https://discuss.ray.io/t/valueerror-rlmodule-config-rlmoduleconfig-has-been-deprecated/21755/1 "2025-02-11T15:11:35Z")
Hello,
I encounter this error while passing the config to PPO:
(MultiAgentEnvRunner pid=28036) File “D:\Miniconda\envs\condaPars\Lib\site-packages\ray\rllib\core\rl\_module\rl\_module.py”, line 113, in build
(MultiAgentEnvRunner pid=28036) module = self.module\_class(module\_config)
(MultiAgentEnvRunner pid=28036) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(MultiAgentEnvRunner pid=28036) deprecation\_warning(
(MultiAgentEnvRunner pid=28036) File “D:\Miniconda\envs\condaPars\Lib\site-packages\ray\rllib\utils\deprecation.py”, line 48, in deprecation\_warning
(MultiAgentEnvRunner pid=28036) raise ValueError(msg)
(MultiAgentEnvRunner pid=28036) ValueError: `RLModule(config=[RLModuleConfig])` has been deprecated. Use `RLModule(observation_space=.., action_space=.., inference_only=.., learner_only=.., model_config=..)` instead.
```
# Create PPO configuration with default policies
self.config = {
"env": "network_env",
"framework": "torch",
"num_gpus": 0,
"num_workers": 2,
"train_batch_size": 4000,
"sgd_minibatch_size": 128,
"lr": 0.0003,
"gamma": 0.99,
"lambda": 0.95,
"use_gae": True,
"disable_env_checking": True,
"_disable_preprocessor_api": True,
"_disable_action_flattening": True,
"experimental": {
"_disable_rl_module": True, # این خط را اضافه کنید
"_validate_config": False,
"enable_rl_module_and_learner": False,
"enable_env_runner_and_connector_v2": False
},
"clip_param": 0.2,
"vf_loss_coeff": 1.0,
"entropy_coeff": 0.0,
"num_sgd_iter": 30,
"rollout_fragment_length": 200,
"batch_mode": "truncate_episodes",
"model": {
"fcnet_hiddens": [256, 256],
"fcnet_activation": "tanh",
},
"multiagent": {
"policies": {
"orchestrator_policy": (None, env.orchestrator_observation_space, env.orchestrator_action_space, {}),
"controller_policy": (None, env.controller_observation_space, env.controller_action_space, {})
},
"policy_mapping_fn": lambda agent_id, *args, **kwargs: (
"orchestrator_policy" if agent_id.startswith("orchestrator")
else "controller_policy"
)
},
"observation_filter": "MeanStdFilter",
"create_env_on_driver": False,
"log_level": "INFO",
# Disable experimental validation and new APIs
"experimental": {
"_validate_config": False,
"enable_rl_module_and_learner": False,
"enable_env_runner_and_connector_v2": False
}
}
# Create trainer
# Create trainer
logger.info("Creating PPO trainer...")
try:
# Initialize trainer
self.trainer = PPO(config=self.config)
logger.info("PPO trainer created successfully")
except Exception as e:
logger.error(f"Error creating PPO trainer: {str(e)}")
raise e
```
How can I address this problem???
---
### Author:  [@christina](https://discuss.ray.io/u/christina)
#### Post date: [February 11, 2025, 8:13pm UTC](https://discuss.ray.io/t/valueerror-rlmodule-config-rlmoduleconfig-has-been-deprecated/21755/2 "2025-02-11T20:13:19Z")
Hello! Welcome to the Ray community 🙂
The error you’re seeing happens because you’re using a deprecated method to initialize the `RLModule` in your PPO configuration. Based on the error message, you should replace `RLModule(config=[RLModuleConfig])` with the new initialization format:
```auto
RLModule(
observation_space=..,
action_space=..,
inference_only=..,
learner_only=..,
model_config=..
)
```
There’s a few diff ways you can resolve it but here are the basic steps:
1. **Update the RLModule Initialization** – Make sure you’re passing the correct parameters when creating an `RLModule`. Instead of the old `config` argument, directly specify `observation_space`, `action_space`, and other necessary parameters.
2. **Enable the New API Stack** – If you’re using an older configuration, migrate to the new API by enabling the RLModule and Learner APIs. You can do this by setting `enable_rl_module_and_learner=True` in your configuration.
3. **Check for Conflicting Settings** – Double-check your config to ensure there are no conflicting parameters. For example, if `_disable_rl_module=True` is set in the `experimental` section, it could be causing issues when using RLModules.
### TL;DR
- Use the **new `RLModule` API** by specifying `observation_space`, `action_space`, and `model_config` directly.
- **Enable the RLModule and Learner APIs** in your configuration to avoid compatibility issues.
- Check for conflicting settings, such as `_disable_rl_module=True`, which may prevent RLModules from working correctly
Here’s some additional reading that might help:
**Docs:**
- [ray.rllib.core.learner.learner.Learner — Ray 2.52.0](https://docs.ray.io/en/latest/rllib/package_ref/doc/ray.rllib.core.learner.learner.Learner.html#ray-rllib-core-learner-learner-learner)
- [https://docs.ray.io/en/latest/rllib/rllib-rlmodule.html#enabling-the-rlmodule-api-in-the-algorithmconfig](https://docs.ray.io/en/latest/rllib/rllib-rlmodule.html#enabling-the-rlmodule-api-in-the-algorithmconfig)
- [ray.rllib.core.rl\_module.rl\_module.RLModule — Ray 2.52.0](https://docs.ray.io/en/latest/rllib/package_ref/doc/ray.rllib.core.rl_module.rl_module.RLModule.html#ray-rllib-core-rl-module-rl-module-rlmodule)
- [https://docs.ray.io/en/latest/rllib/rllib-rlmodule.html#default-rlmodules](https://docs.ray.io/en/latest/rllib/rllib-rlmodule.html#default-rlmodules)
---
### Author:  [@Ali\_Zargarian](https://discuss.ray.io/u/Ali_Zargarian)
#### Post date: [March 18, 2025, 8:53am UTC](https://discuss.ray.io/t/valueerror-rlmodule-config-rlmoduleconfig-has-been-deprecated/21755/3 "2025-03-18T08:53:51Z")
Hi Christina,
I face the same error when I tried to train the example:cartpole\_with\_dict\_observation\_space.py
would you mind explain more how it should be set the config and RLModule?
unfortunately I didn’t find an standard documents or a simple example for solving this problem! especially by new API stack.
my config is something as below :
```
config = (
PPOConfig()
.api_stack(
enable_rl_module_and_learner=True,
enable_env_runner_and_connector_v2=True,
)
.environment(
env="CartPoleWithDictObservationSpace",
)
.training(
lr = agent_params["lr"], # Learning rate
entropy_coeff= agent_params["entropy_coeff"], # Encourage exploration with entropy regularization
)
.env_runners(num_env_runners = agent_params["num_env"]) # Number of parallel environments
.framework("torch")
.rl_module(
rl_module_spec=RLModuleSpec(
module_class=PPOTorchRLModule,
inference_only=False,
learner_only=False,
observation_space=env_instance.observation_space,
action_space=env_instance.action_space,
model_config={"hidden_sizes": [128, 128]}, # Optional, passed to RLModule
)
)
)
```
thanks in advanced
---
### Author:  [@kmprasad4u](https://discuss.ray.io/u/kmprasad4u)
#### Post date: [April 13, 2025, 4:08pm UTC](https://discuss.ray.io/t/valueerror-rlmodule-config-rlmoduleconfig-has-been-deprecated/21755/4 "2025-04-13T16:08:10Z")
I also faced the below error when trying out, and found that the issue I had is because I was importing “override” from typing module. Instead if i use from ray’s library, it is working. Not sure if this is the right way and this is intentional
ray.rllib.utils.annotations import override
`RLModule(config=[RLModuleConfig])` has been deprecated. Use `RLModule(observation_space=.., action_space=.., inference_only=.., learner_only=.., model_config=..)` instead.
---
### Author:  [@Ali\_Zargarian](https://discuss.ray.io/u/Ali_Zargarian)
#### Post date: [April 18, 2025, 2:35pm UTC](https://discuss.ray.io/t/valueerror-rlmodule-config-rlmoduleconfig-has-been-deprecated/21755/5 "2025-04-18T14:35:46Z")
Hi @kmprasad4u
interesting. Could you please share more info?
How you set your config?
Thanks
---
### Author:  [@kmprasad4u](https://discuss.ray.io/u/kmprasad4u)
#### Post date: [April 19, 2025, 5:19am UTC](https://discuss.ray.io/t/valueerror-rlmodule-config-rlmoduleconfig-has-been-deprecated/21755/6 "2025-04-19T05:19:57Z")
Edit: I see that you are using Ray’s PPOTorchRLModule and not a custom module. Then i think this may not help you. Ensure you have latest packages
This is not related to config. Below a snippet. There may be other reasons for the same error. Ensure you are using New API stack and passing the correct config as well
```auto
from typing import Any, Dict
from ray.rllib.algorithms.ppo import PPOConfig
from ray.rllib.core.rl_module.rl_module import RLModuleSpec
from ray.rllib.core.rl_module.torch.torch_rl_module import TorchRLModule
from ray.rllib.core.rl_module.apis.value_function_api import ValueFunctionAPI
from ray.rllib.utils.annotations import override
class NeuralNetworkModel(TorchRLModule, ValueFunctionAPI):
def setup(self):
# layer_dimensions = self.model_config["layer_dimensions"]
pass
# the override here, I was using "from typing". Changing it to from ray.rllib.utils.annotations resolved the issue for me.
@override(TorchRLModule)
def _forward(self, batch: Dict[str, Any], **kwargs) -> Dict[str, Any]:
pass
config = (
PPOConfig()
.rl_module(
rl_module_spec = RLModuleSpec(
module_class = NeuralNetworkModel,
model_config={
# "layer_dimensions": (32, 16, 8)
}
)
)
)
```