# Custom Multi ENV with DQN: AttributeError: 'NoneType' object has no attribute 'build\_encoder'

**URL:** <https://discuss.ray.io/t/custom-multi-env-with-dqn-attributeerror-nonetype-object-has-no-attribute-build-encoder/21711>\
**Category:** RLlib\
**Created:** [February 7, 2025, 9:49am UTC](https://discuss.ray.io/t/custom-multi-env-with-dqn-attributeerror-nonetype-object-has-no-attribute-build-encoder/21711 "2025-02-07T09:49:53Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Advertise-Here](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/advertise-here/32/7644_2.png) [@Advertise-Here](https://discuss.ray.io/u/Advertise-Here)\
**Post date:** [February 7, 2025, 9:49am UTC](https://discuss.ray.io/t/custom-multi-env-with-dqn-attributeerror-nonetype-object-has-no-attribute-build-encoder/21711/1 "2025-02-07T09:49:53Z")

</div>

**How severe does this issue affect your experience of using Ray?**

- High: It blocks me to complete my task.

I have a custom Multi Agent Environment inheriting from MultiAgentEnv. Simply, it has two agents who must guess a number from 1 to 100. The environment also picks a number from 1 to 100 (always 42). If the agent gets it correct, reward +1, 0 otherwise.

When building my algorithm with DQN (or really any other algorithm), I get the following error:

> File “/rayTest/.venv/lib/python3.12/site-packages/ray/rllib/algorithms/dqn/default\_dqn\_rl\_module.py”, line 52, in setup  
> self.encoder = self.catalog.build\_encoder(framework=self.framework)  
> ^^^^^^^^^^^^^^^^^^^^^^^^^^  
> AttributeError: ‘NoneType’ object has no attribute ‘build\_encoder’

To me it appears to the catalog (?) not being defined. I have tried forcing this explicitly to no avail 😢

My config is as follows:

```auto
from ray.rllib.algorithms.dqn import DQNConfig
from ray import train
from ray.rllib.algorithms.dqn.dqn_catalog import DQNCatalog

config = (
    DQNConfig()
    .environment(env=HI_Environment, env_config=env_config)
    .multi_agent(
        policies={"policy_alice", "policy_bob"},
        policy_mapping_fn=lambda agent_id: f"policy_{agent_id}",
        policies_to_train=["policy_alice", "policy_bob"],
    )
    .rl_module(
        rl_module_spec=MultiRLModuleSpec(rl_module_specs={
            "policy_alice": RLModuleSpec(observation_space=Discrete(100), action_space=Discrete(100)),
            "policy_bob": RLModuleSpec(observation_space=Discrete(100), action_space=Discrete(100)),
            
        }),
    )
    .api_stack(
        enable_rl_module_and_learner=True,
        enable_env_runner_and_connector_v2=True,
    )
    .framework("torch")
    .env_runners(num_env_runners=1)
    .training(dueling=False, double_q=False)
)

algo = config.build()

```

Running Python 3.12.8 with  
pygame == 2.6.1  
pettingzoo == 1.24.3  
tqdm == 4.67.1 # progress bar  
ray[train,tune,rllib,default] == 2.41.0  
pillow == 11.1.0  
torch == 2.5.1  
ipykernel == 6.29.5  
ipywidgets == 8.1.5

Appreciate any guidance on what is likely something obvious except to me!

thanks

---

<div class="post-metadata">

**Author:** ![Lars\_Simon\_Zehnder](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/lars_simon_zehnder/32/1185_2.png) [@Lars\_Simon\_Zehnder](https://discuss.ray.io/u/Lars_Simon_Zehnder)\
**Post date:** [February 11, 2025, 5:53pm UTC](https://discuss.ray.io/t/custom-multi-env-with-dqn-attributeerror-nonetype-object-has-no-attribute-build-encoder/21711/2 "2025-02-11T17:53:03Z")

</div>

Hi @Advertise-Here , thanks for raising this issue and welcome to the forum.

When giving a custom module you need to provide a [`ray.rllib.core.models.catalog.Catalog`](https://discuss.ray.io/t/using-dict-observation-space-with-custom-rlmodule/20892) The definition of a custom `RLModule` is here, however, not required - as long as you do not want to use a custom module you programmed.

So, if you could try again without using the `.rl_module` method in the configuration, this should work:

```auto
config = (
    DQNConfig()
    .environment(env=HI_Environment, env_config=env_config)
    .multi_agent(
        policies={"policy_alice", "policy_bob"},
        policy_mapping_fn=lambda agent_id: f"policy_{agent_id}",
        policies_to_train=["policy_alice", "policy_bob"],
    )
    .api_stack(
        enable_rl_module_and_learner=True,
        enable_env_runner_and_connector_v2=True,
    )
    .framework("torch")
    .env_runners(num_env_runners=1)
    .training(dueling=False, double_q=False)
)
```