### Author:  [@Ali\_Zargarian](https://discuss.ray.io/u/Ali_Zargarian)
#### Post date: [March 21, 2025, 8:31am UTC](https://discuss.ray.io/t/valueerror-rlmodule-config-rlmoduleconfig-has-been-deprecated-new-api-stack/22104/1 "2025-03-21T08:31:52Z")
High: Completely blocks me.
Hello,
During the training my costume environment in gym with Rllib(new API Stack) I get this error and couldn’t solve it up to now 🙁 I have defined my observations as Dictionary. It works well by old API Stack but not with new one. It completely blocks me to go forward. In order to have a better and simpler understanding I started to train the example in Rllib github in below link:
>
Unfortunately I get the same error. as clarification I define the config as below :
config = (
PPOConfig()
.api\_stack(
enable\_rl\_module\_and\_learner=True,
enable\_env\_runner\_and\_connector\_v2=True,
)
.environment(
env=“CartPoleWithDictObservationSpace”,
)
.training(
lr = agent\_params[“lr”], # Learning rate
entropy\_coeff= agent\_params[“entropy\_coeff”], # Encourage exploration with entropy regularization
)
.env\_runners(num\_env\_runners = agent\_params[“num\_env”]) # Number of parallel environments
.framework(“torch”)
```
.rl_module(
rl_module_spec=RLModuleSpec(
module_class=PPOTorchRLModule,
inference_only=False,
learner_only=False,
observation_space=env_instance.observation_space,
action_space=env_instance.action_space,
model_config={"hidden_sizes": [128, 128]}, # Optional, passed to RLModule
)
)
```
Environment:
- Ray version: 2.43.0
- Python version: 3.12.9
thanks
---
### Author:  [@christina](https://discuss.ray.io/u/christina)
#### Post date: [March 25, 2025, 12:59am UTC](https://discuss.ray.io/t/valueerror-rlmodule-config-rlmoduleconfig-has-been-deprecated-new-api-stack/22104/2 "2025-03-25T00:59:54Z")
Hi Ali! Have you had a chance to look at the old → new API migration guide? [New API stack migration guide — Ray 2.44.0](https://docs.ray.io/en/latest/rllib/new-api-stack-migration-guide.html#)
I took a look and it seems like someone on Github opened up a similar issue, is it similar to the one you’re experiencing? [RLlib new API stack false deprecation warning / MultiRLModuleSpec · Issue #51630 · ray-project/ray · GitHub](https://github.com/ray-project/ray/issues/51630) 🤔
---
### Author:  [@Ali\_Zargarian](https://discuss.ray.io/u/Ali_Zargarian)
#### Post date: [March 25, 2025, 8:40am UTC](https://discuss.ray.io/t/valueerror-rlmodule-config-rlmoduleconfig-has-been-deprecated-new-api-stack/22104/3 "2025-03-25T08:40:13Z")
Hi Christina ,
firstly , thanks for your reply.
yes, I’ve seen that lots of times. As you see I arranged the rlmodule according the new API’s arrangement and also warning deprecation. I think New API module can not handle the dictionary observation. Actually I couldn’t find some resources for training the agent with dict observation and New API stack 🙁 . For some reasons I need to define my observations as dictionary and I’m eager to train it with new API stack. I would be so appreciable if you give me some hints or resources to address the issue more direct.
Thanks
---
### Author:  [@jiangpeng\_li](https://discuss.ray.io/u/jiangpeng_li)
#### Post date: [March 26, 2025, 2:42am UTC](https://discuss.ray.io/t/valueerror-rlmodule-config-rlmoduleconfig-has-been-deprecated-new-api-stack/22104/4 "2025-03-26T02:42:18Z")
Hi. Have you tried to remove that
```python
observation_space=env_instance.observation_space,
action_space=env_instance.action_space,
```
from the `RLModuleSpec()` definition? It seems that those configurations can be figured out by the `RLModuleSpec` itself.
---
### Author:  [@sven1977](https://discuss.ray.io/u/sven1977)
#### Post date: [March 26, 2025, 8:34am UTC](https://discuss.ray.io/t/valueerror-rlmodule-config-rlmoduleconfig-has-been-deprecated-new-api-stack/22104/5 "2025-03-26T08:34:07Z")
Hey @Ali_Zargarian , could you provide some minimal reproduction script, so we can debug this problem on our end?
Thanks!
---
### Author:  [@Ali\_Zargarian](https://discuss.ray.io/u/Ali_Zargarian)
#### Post date: [March 26, 2025, 11:15am UTC](https://discuss.ray.io/t/valueerror-rlmodule-config-rlmoduleconfig-has-been-deprecated-new-api-stack/22104/6 "2025-03-26T11:15:59Z")
Hi @sven1977 ,
Thanks for rapid reply 🙂
As I said I took the environment from ray example Github as below:
>
and my training script is as below:
```auto
from cartpole_with_dict_observation_space import CartPoleWithDictObservationSpace
def env_creator(config):
return CartPoleWithDictObservationSpace()
# Register the custom environment
register_env("CartPoleWithDictObservationSpace", env_creator)
env_instance = CartPoleWithDictObservationSpace()
config = (
PPOConfig()
.api_stack(
enable_rl_module_and_learner=True,
enable_env_runner_and_connector_v2=True,
)
.environment(
env="CartPoleWithDictObservationSpace",
)
.training(
lr = 0.0001, # Learning rate
entropy_coeff= 0.01, # Encourage exploration with entropy regularization
)
.env_runners(num_env_runners = 1) # Number of parallel environments
.framework("torch")
)
RLModule(
rl_module_spec=RLModuleSpec(
module_class=PPOTorchRLModule,
inference_only=False,
learner_only=False,
observation_space=env_instance.observation_space,
action_space=env_instance.action_space,
model_config={"hidden_sizes": [128, 128]}, # Optional, passed to RLModule
catalog_class=PPOCatalog,
)
)
tuner = tune.Tuner(
"PPO",
param_space=config,
run_config=tune.RunConfig(
stop={"training_iteration": 5}, # Specify when to stop training
checkpoint_config=train.CheckpointConfig(checkpoint_at_end=True),
),
)
# Run the tuner
results = tuner.fit()
# Get the best result based on a particular metric
best_result = results.get_best_result(
metric="env_runners/episode_return_mean", mode="max"
)
```
Thanks
---
### Author:  [@Ali\_Zargarian](https://discuss.ray.io/u/Ali_Zargarian)
#### Post date: [March 26, 2025, 2:53pm UTC](https://discuss.ray.io/t/valueerror-rlmodule-config-rlmoduleconfig-has-been-deprecated-new-api-stack/22104/7 "2025-03-26T14:53:39Z")
Unfortunately, it doesn’t work. Also it should be there according to deprecation warning.
---
### Author:  [@Ali\_Zargarian](https://discuss.ray.io/u/Ali_Zargarian)
#### Post date: [April 2, 2025, 9:24am UTC](https://discuss.ray.io/t/valueerror-rlmodule-config-rlmoduleconfig-has-been-deprecated-new-api-stack/22104/8 "2025-04-02T09:24:36Z")
Hello @sven1977 ,
any update? 😉
---
### Author:  [@Ali\_Zargarian](https://discuss.ray.io/u/Ali_Zargarian)
#### Post date: [April 24, 2025, 2:33pm UTC](https://discuss.ray.io/t/valueerror-rlmodule-config-rlmoduleconfig-has-been-deprecated-new-api-stack/22104/9 "2025-04-24T14:33:31Z")
Hello @sven1977
is there any update? As I’m working on another issue related to masking some actions, something as below :
>
it uses Dict observations again. The example explains that is suitable just to train with Old API stack .Because I don’t want to use Old API stack, I’ve been blocked again!!
Thanks in advanced
---
### Author:  [@Lars\_Simon\_Zehnder](https://discuss.ray.io/u/Lars_Simon_Zehnder)
#### Post date: [April 24, 2025, 2:40pm UTC](https://discuss.ray.io/t/valueerror-rlmodule-config-rlmoduleconfig-has-been-deprecated-new-api-stack/22104/10 "2025-04-24T14:40:52Z")
@Ali_Zargarian , this is indeed the old example. The example for the new API stack can be found here: [ray/rllib/examples/rl\_modules/action\_masking\_rl\_module.py at master · ray-project/ray · GitHub](https://github.com/ray-project/ray/blob/master/rllib/examples/rl_modules/action_masking_rl_module.py)
---
### Author:  [@Ali\_Zargarian](https://discuss.ray.io/u/Ali_Zargarian)
#### Post date: [April 24, 2025, 2:47pm UTC](https://discuss.ray.io/t/valueerror-rlmodule-config-rlmoduleconfig-has-been-deprecated-new-api-stack/22104/11 "2025-04-24T14:47:06Z")
@sven1977
Ah.., Nice,
could you please take a look at example for train(or tune) the observation as dictionary(DictObservationSpace) with New API stack?
Thanks
---
### Author:  [@Lars\_Simon\_Zehnder](https://discuss.ray.io/u/Lars_Simon_Zehnder)
#### Post date: [April 24, 2025, 2:49pm UTC](https://discuss.ray.io/t/valueerror-rlmodule-config-rlmoduleconfig-has-been-deprecated-new-api-stack/22104/12 "2025-04-24T14:49:33Z")
@Ali_Zargarian , I don’t understand what you mean exactly. Could you elaborate? The `observation_space` in action masking is always dictionary. Do you mean using a dictionary observation space under the `observations` key?
---
### Author:  [@Ali\_Zargarian](https://discuss.ray.io/u/Ali_Zargarian)
#### Post date: [May 2, 2025, 2:57pm UTC](https://discuss.ray.io/t/valueerror-rlmodule-config-rlmoduleconfig-has-been-deprecated-new-api-stack/22104/15 "2025-05-02T14:57:06Z")
Hi @Lars_Simon_Zehnder ,
I tried to use a dictionary observation space under the `observations` key , but I get this error :
(PPO pid=10788) ValueError: This RLModule requires the environment to provide a `gym.spaces.Dict` observation space of the form: [repeated 2x across cluster]
(PPO pid=10788) {‘action\_mask’: Box(0.0, 1.0, shape=(self.action\_space.n,)), ‘observation\_space’: self.observation\_space} [repeated 2x across cluster]
in spite of I get this when I print observation\_space’s type :
\
observation\_space is something as below :
```
import gymnasium as gym
self.observation_space = gym.spaces.Dict({
"action_mask": gym.spaces.Box(
low=0,
high=1,
shape=(self.action_space.n,),
dtype=np.float32
),
"observations": gym.spaces.Dict({
"obs_1": gym.spaces.Box(
low=-100.0,
high=100.0,
shape=(4 * 4,),
dtype=np.float32
),
"obs_2": gym.spaces.Box(
low=-100.0,
high=100.0,
shape=(3,),
dtype=np.float32
)
})
})
```
I think the action\_masking\_rlm.py can’t handle nested “observations” dictionary!
---
### Author:  [@Ali\_Zargarian](https://discuss.ray.io/u/Ali_Zargarian)
#### Post date: [May 5, 2025, 4:38pm UTC](https://discuss.ray.io/t/valueerror-rlmodule-config-rlmoduleconfig-has-been-deprecated-new-api-stack/22104/16 "2025-05-05T16:38:13Z")
Hi @Lars_Simon_Zehnder
any update?
---
### Author:  [@Daraan](https://discuss.ray.io/u/Daraan)
#### Post date: [June 3, 2025, 1:16pm UTC](https://discuss.ray.io/t/valueerror-rlmodule-config-rlmoduleconfig-has-been-deprecated-new-api-stack/22104/17 "2025-06-03T13:16:25Z")
I only skimmed trough your topic. If it is the same warning message I am aware of you can safely ignore it. Currently there will _always_ be a deprecation warning for the very first `RLModuleConfig` config created.
There still exists deprecated code that creates a `self.config = RLModuleConfig(...` from your new stack options for _backwards compatibility_, this instantiation throws the error
I wrote myself a snippet to run before my code to suppress this misleading warning
```python
from ray.rllib.utils.deprecation import logger as __deprecation_logger
# This suppresses a deprecation warning from RLModuleConfig
__old_level =__ deprecation_logger.getEffectiveLevel()
__deprecation_logger.setLevel(logging.ERROR)
RLModuleConfig()
__deprecation_logger.setLevel(__old_level)
del __deprecation_logger
```
Try if that hides the error. To be sure run it with a debugger and find the location, it could be a false postive and unrelated to your code.