# Custom Model encode attribute error

**URL:** <https://discuss.ray.io/t/custom-model-encode-attribute-error/9283>\
**Category:** RLlib\
**Created:** [February 7, 2023, 8:27pm UTC](https://discuss.ray.io/t/custom-model-encode-attribute-error/9283 "2023-02-07T20:27:13Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![evo11x](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/evo11x/32/2303_2.png) [@evo11x](https://discuss.ray.io/u/evo11x)\
**Post date:** [February 7, 2023, 8:27pm UTC](https://discuss.ray.io/t/custom-model-encode-attribute-error/9283/1 "2023-02-07T20:27:13Z")

</div>

- High: It blocks me to complete my task.

I have this custom model which process two observations, one with CNN and one with dense

```auto
class CustomModel(TorchModelV2, nn.Module):
    def __init__ (self, obs_space, action_space, num_outputs, model_config, name):
        TorchModelV2. __init__ (
            self, obs_space, action_space, num_outputs, model_config, name
        )
        nn.Module. __init__ (self)
        self.observation_space = obs_space
        self.action_space = action_space
        
        self.conv = nn.Conv2d(1, 32, kernel_size=3, stride=1)
        self.fc1 = nn.Linear(71*30*32, 256)
        self.fc2 = nn.Linear(256 + 12, 64)
        self.fc3 = nn.Linear(64, action_space.shape[0])
        
    def forward(self, observation, state, mask):
        obs1 = observation["obs1"]
        obs2 = observation["obs2"]
        
        obs1 = obs1.unsqueeze(0).unsqueeze(0)
        obs1 = F.relu(self.conv(obs1))
        obs1 = obs1.view(-1, 71*30*32)
        obs1 = F.relu(self.fc1(obs1))
        
        x = torch.cat([obs1, obs2], dim=1)
        x = F.relu(self.fc2(x))
        x = self.fc3(x)
        
        return x, state

```

And I get this error about encode attribute:

```auto
Traceback (most recent call last):
  File "D:\TestProj\kub_IMPALA.py", line 234, in <module>
    trainer = ppo.APPOTrainer(config=cfg)
  File "D:\TestProj\.env\lib\site-packages\ray\rllib\algorithms\appo\appo.py", line 201, in __init__
    super(). __init__ (config, *args, **kwargs)
  File "D:\TestProj\.env\lib\site-packages\ray\rllib\algorithms\algorithm.py", line 441, in __init__
    super(). __init__ (
  File "D:\TestProj\.env\lib\site-packages\ray\tune\trainable\trainable.py", line 169, in __init__  
    self.setup(copy.deepcopy(self.config))
  File "D:\TestProj\.env\lib\site-packages\ray\rllib\algorithms\appo\appo.py", line 210, in setup   
    super().setup(config)
  File "D:\TestProj\.env\lib\site-packages\ray\rllib\algorithms\impala\impala.py", line 464, in setup
    super().setup(config)
  File "D:\TestProj\.env\lib\site-packages\ray\rllib\algorithms\algorithm.py", line 566, in setup   
    self.workers = WorkerSet(
  File "D:\TestProj\.env\lib\site-packages\ray\rllib\evaluation\worker_set.py", line 191, in __init__
    raise e.args[0].args[2]
AttributeError: 'CustomModel' object has no attribute 'encode'
(RolloutWorker pid=558772) 2023-02-07 22:17:11,837 ERROR worker.py:763 -- Exception raised in creation task: The actor died because of an error raised in its creation task, ray::RolloutWorker. __init__ () (pid=558772, ip=127.0.0.1, repr=<ray.rllib.evaluation.rollout_worker.RolloutWorker object at 0x0000019A9D35CAC0>)
(RolloutWorker pid=558772) File "python\ray\_raylet.pyx", line 830, in ray._raylet.execute_task
(RolloutWorker pid=558772) File "python\ray\_raylet.pyx", line 834, in ray._raylet.execute_task
(RolloutWorker pid=558772) File "python\ray\_raylet.pyx", line 780, in ray._raylet.execute_task.function_executor        
(RolloutWorker pid=558772) File "D:\TestProj\.env\lib\site-packages\ray\_private\function_manager.py", line 674, in actor_method_executor
(RolloutWorker pid=558772) return method(__ray_actor, *args, **kwargs)
(RolloutWorker pid=558772) File "D:\TestProj\.env\lib\site-packages\ray\util\tracing\tracing_helper.py", line 466, in _resume_span
(RolloutWorker pid=558772) return method(self, *_args, **_kwargs)
(RolloutWorker pid=558772) File "D:\TestProj\.env\lib\site-packages\ray\rllib\evaluation\rollout_worker.py", line 712, in __init__
(RolloutWorker pid=558772) self._build_policy_map(
(RolloutWorker pid=558772) File "D:\TestProj\.env\lib\site-packages\ray\util\tracing\tracing_helper.py", line 466, in _resume_span
(RolloutWorker pid=558772) return method(self, *_args, **_kwargs)
(RolloutWorker pid=558772) File "D:\TestProj\.env\lib\site-packages\ray\rllib\evaluation\rollout_worker.py", line 1970, in _build_policy_map
(RolloutWorker pid=558772) self.policy_map.create_policy(
(RolloutWorker pid=558772) File "D:\TestProj\.env\lib\site-packages\ray\rllib\policy\policy_map.py", line 146, in create_policy
(RolloutWorker pid=558772) policy = create_policy_for_framework(
(RolloutWorker pid=558772) File "D:\TestProj\.env\lib\site-packages\ray\rllib\utils\policy.py", line 126, in create_policy_for_framework
(RolloutWorker pid=558772) return policy_class(observation_space, action_space, merged_config)
(RolloutWorker pid=558772) File "D:\TestProj\.env\lib\site-packages\ray\rllib\algorithms\appo\appo_torch_policy.py", line 78, in __init__
(RolloutWorker pid=558772) TorchPolicyV2. __init__ (
(RolloutWorker pid=558772) File "D:\TestProj\.env\lib\site-packages\ray\rllib\policy\torch_policy_v2.py", line 88, in __init__
(RolloutWorker pid=558772) model, dist_class = self._init_model_and_dist_class()
(RolloutWorker pid=558772) File "D:\TestProj\.env\lib\site-packages\ray\rllib\policy\torch_policy_v2.py", line 446, in _init_model_and_dist_class
(RolloutWorker pid=558772) value = _internal_kv_get(_make_key(self._prefix, category, key))
(RolloutWorker pid=558772) File "D:\TestProj\.env\lib\site-packages\ray\tune\registry.py", line 166, in _make_key
(RolloutWorker pid=558772) + key.encode("ascii")
(RolloutWorker pid=558772) File "D:\TestProj\.env\lib\site-packages\torch\nn\modules\module.py", line 1269, in __getattr__
(RolloutWorker pid=558772) raise AttributeError("'{}' object has no attribute '{}'".format(
(RolloutWorker pid=558772) AttributeError: 'CustomModel' object has no attribute 'encode'

```

this is the initialization

```auto
    env = MyEnv()
    policy_network = CustomModel(
        obs_space=env.observation_space,
        action_space=env.action_space,
        num_outputs=env.action_space.shape[0],
        model_config={},
        name="CustomModel"
    )

```

---

<div class="post-metadata">

**Author:** ![mannyv](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/mannyv/32/606_2.png) [@mannyv](https://discuss.ray.io/u/mannyv)\
**Post date:** [February 7, 2023, 9:44pm UTC](https://discuss.ray.io/t/custom-model-encode-attribute-error/9283/2 "2023-02-07T21:44:46Z")

</div>

Hi @evo11x,

Can you share the config you are using?

---

<div class="post-metadata">

**Author:** ![evo11x](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/evo11x/32/2303_2.png) [@evo11x](https://discuss.ray.io/u/evo11x)\
**Post date:** [February 7, 2023, 10:12pm UTC](https://discuss.ray.io/t/custom-model-encode-attribute-error/9283/3 "2023-02-07T22:12:50Z")

</div>

@mannyv this is the config

```auto
cfg = ppo.appo.DEFAULT_CONFIG.copy()
cfg_custom = {
        "env": MyEnv,
        "num_gpus": 1,
        "num_workers": 5,
        "num_envs_per_worker": 15,
        "framework": "torch",
        "horizon": 500,
        "lr": 0.0000001, # 0.000005        
        "momentum": 0.9,        
        "num_sgd_iter": 30,
        "train_batch_size": 20000,        
        "model": {
          "custom_model": policy_network
        },        
        "evaluation_duration_unit": "episodes",
        "evaluation_num_workers": 1,
        "evaluation_interval": 50,        
    }    
    cfg.update(cfg_custom)

```

---

<div class="post-metadata">

**Author:** ![evo11x](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/evo11x/32/2303_2.png) [@evo11x](https://discuss.ray.io/u/evo11x)\
**Post date:** [February 7, 2023, 10:14pm UTC](https://discuss.ray.io/t/custom-model-encode-attribute-error/9283/4 "2023-02-07T22:14:27Z")

</div>

And this is the observation

```auto
self.observation_space = gym.spaces.Dict({          
            "obs1": gym.spaces.Box(low=0.0, high=255.0, shape=(71,30), dtype=np.float32),
            "obs2": gym.spaces.Box(low=-np.inf, high=np.inf, shape=(12,), dtype=np.float32)            
        })

```

---

<div class="post-metadata">

**Author:** ![mannyv](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/mannyv/32/606_2.png) [@mannyv](https://discuss.ray.io/u/mannyv)\
**Post date:** [February 7, 2023, 10:43pm UTC](https://discuss.ray.io/t/custom-model-encode-attribute-error/9283/5 "2023-02-07T22:43:44Z")

</div>

Hi @evo11x,

I think it should be:

```auto
model": {
          "custom_model": CustomModel
        },  

```

---

<div class="post-metadata">

**Author:** ![evo11x](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/evo11x/32/2303_2.png) [@evo11x](https://discuss.ray.io/u/evo11x)\
**Post date:** [February 8, 2023, 1:05am UTC](https://discuss.ray.io/t/custom-model-encode-attribute-error/9283/6 "2023-02-08T01:05:57Z")

</div>

> [@mannyv](#):
>
> `"custom_model": CustomModel`

Thanks!, now I get this error

```auto
in forward
    obs1 = observation['obs1']
  File "d:\TestProj\.env\lib\site-packages\ray\rllib\policy\sample_batch.py", line 818, in __getitem__
    value = dict. __getitem__ (self, key)
KeyError: 'obs1'

```

the observation key is ok in the environment, but I don’t know how to access it from the custom model

---

<div class="post-metadata">

**Author:** ![evo11x](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/evo11x/32/2303_2.png) [@evo11x](https://discuss.ray.io/u/evo11x)\
**Post date:** [February 8, 2023, 1:21am UTC](https://discuss.ray.io/t/custom-model-encode-attribute-error/9283/7 "2023-02-08T01:21:56Z")

</div>

Ok, I found how to access the obs1,  
observation[‘obs’][‘obs1’]

but there is also an item ‘obs\_new’ in observation which I don’t know what it is

---

<div class="post-metadata">

**Author:** ![mannyv](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/mannyv/32/606_2.png) [@mannyv](https://discuss.ray.io/u/mannyv)\
**Post date:** [February 8, 2023, 1:49am UTC](https://discuss.ray.io/t/custom-model-encode-attribute-error/9283/8 "2023-02-08T01:49:22Z")

</div>

Do you mean `new_obs`?

This would be the during the training steps but not the rollout steps.

> <https://github.com/ray-project/ray/blob/5e1def0a95140a01c869c10ea8ec943c8280c325/rllib/policy/sample_batch.py#L115>
