# Attribute Error when using Custom gymnasium Environment

**URL:** <https://discuss.ray.io/t/attribute-error-when-using-custom-gymnasium-environment/21782>\
**Category:** RLlib\
**Created:** [February 12, 2025, 11:14pm UTC](https://discuss.ray.io/t/attribute-error-when-using-custom-gymnasium-environment/21782 "2025-02-12T23:14:45Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![Dekermanjian](https://avatars.discourse-cdn.com/v4/letter/d/96bed5/32.png) [@Dekermanjian](https://discuss.ray.io/u/Dekermanjian)\
**Post date:** [February 12, 2025, 11:14pm UTC](https://discuss.ray.io/t/attribute-error-when-using-custom-gymnasium-environment/21782/1 "2025-02-12T23:14:45Z")

</div>

**How severe does this issue affect your experience of using Ray?**

- High: It blocks me to complete my task.

I am learning how to use Ray and the book I am using was written using an older version or Ray. I am currently running into an issue with RLlib where the problem seems to be stemming from using a Custom Environment. The code errors out with a `AttributeError: 'NoneType' object has no attribute 'build_encoder'`. It seems this is an issue with building a catalog but at this stage of my learning of ray I don’t really know what that is. Any help would be appreciated. Here is the code I am using:

```auto
import gymnasium as gym
from gymnasium.spaces import Discrete
import os
from ray.tune.logger import pretty_print
from ray.rllib.algorithms.dqn import DQNConfig
from ray.tune.registry import register_env

class MyEnvironment(gym.Env):

    seeker, goal = (0, 0), (4, 4)
    info = {'seeker': seeker, 'goal': goal}

    def __init__ (self, config=None):
        self.action_space = Discrete(4)
        self.observation_space = Discrete(5*5)

    def reset(self, seed=None, options=None):
        """Reset seeker and goal positions, return observations."""
        self.seeker = (0, 0)
        self.goal = (4, 4)

        return self.get_observation(), {}

    def get_observation(self):
        """Encode the seeker position as integer"""
        return 5 * self.seeker[0] + self.seeker[1]

    def get_reward(self):
        """Reward finding the goal"""
        return 1 if self.seeker == self.goal else 0

    def is_done(self):
        """We're done if we found the goal"""
        return self.seeker == self.goal

    def step(self, action):
        """Take a step in a direction and return all available information."""
        if action == 0: # move down
            self.seeker = (min(self.seeker[0] + 1, 4), self.seeker[1])
        elif action == 1: # move left
            self.seeker = (self.seeker[0], max(self.seeker[1] - 1, 0))
        elif action == 2: # move up
            self.seeker = (max(self.seeker[0] - 1, 0), self.seeker[1])
        elif action == 3: # move right
            self.seeker = (self.seeker[0], min(self.seeker[1] + 1, 4))
        else:
            raise ValueError("Invalid action")

        return self.get_observation()[0], self.get_reward(), self.is_done(), False, self.info

    def render(self, *args, **kwargs):
        """Render the environment, e.g. by printing its representation."""
        os.system('cls' if os.name == 'nt' else 'clear')
        try:
            from IPython.display import clear_output
            clear_output(wait=True)
        except Exception:
            pass
        grid = [['| ' for _ in range(5)] + ["|\n"] for _ in range(5)]
        grid[self.goal[0]][self.goal[1]] = '|G'
        grid[self.seeker[0]][self.seeker[1]] = '|S'
        print(''.join([''.join(grid_row) for grid_row in grid]))

config = (
    DQNConfig()
      .environment(MyEnvironment, env_config={})
      .env_runners(
          num_env_runners = 2, 
          create_env_on_local_worker = True, 
      )
      .api_stack(
          enable_rl_module_and_learner=True, 
          enable_env_runner_and_connector_v2=True
      )
)

algo = config.build_algo()

```

---

<div class="post-metadata">

**Author:** ![mannyv](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/mannyv/32/606_2.png) [@mannyv](https://discuss.ray.io/u/mannyv)\
**Post date:** [February 12, 2025, 11:24pm UTC](https://discuss.ray.io/t/attribute-error-when-using-custom-gymnasium-environment/21782/2 "2025-02-12T23:24:09Z")

</div>

@Dekermanjian,

Would it be easier to downgrade to the same version of Ray the book uses?

---

<div class="post-metadata">

**Author:** ![Dekermanjian](https://avatars.discourse-cdn.com/v4/letter/d/96bed5/32.png) [@Dekermanjian](https://discuss.ray.io/u/Dekermanjian)\
**Post date:** [February 12, 2025, 11:26pm UTC](https://discuss.ray.io/t/attribute-error-when-using-custom-gymnasium-environment/21782/3 "2025-02-12T23:26:46Z")

</div>

Thank you for the quick response, @mannyv.

Yes that would certainly be an option and it would be the simplest option. But then that would mean that I would always have to use a downgraded version of Ray because I don’t know how to work around the error with a current version of Ray.

---

<div class="post-metadata">

**Author:** ![mannyv](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/mannyv/32/606_2.png) [@mannyv](https://discuss.ray.io/u/mannyv)\
**Post date:** [February 13, 2025, 1:00am UTC](https://discuss.ray.io/t/attribute-error-when-using-custom-gymnasium-environment/21782/4 "2025-02-13T01:00:21Z")

</div>

@Dekermanjian,

I guess it sorry of depends on your learning objectives. If you want to learn about reinforcement learning, in a hands on way, and ticket with how the pieces got together then it really doesn’t matter which version you use.

On the other hand, the RLLIB implementation and core abstractions have changed drastically in the past two years and are still partway through that migration. You could expect to spend a lot of time figuring out how the new pieces match up with the old ones and how to convert everything. There is also a lot of valuable learning and skills to develop in doing that but those are not really reinforcement learning related.

---

<div class="post-metadata">

**Author:** ![Dekermanjian](https://avatars.discourse-cdn.com/v4/letter/d/96bed5/32.png) [@Dekermanjian](https://discuss.ray.io/u/Dekermanjian)\
**Post date:** [February 13, 2025, 1:10am UTC](https://discuss.ray.io/t/attribute-error-when-using-custom-gymnasium-environment/21782/5 "2025-02-13T01:10:28Z")

</div>

Thank you @mannyv for that information. The book I am going through is about how to scale workloads with Ray. It walks through the different components of Ray and discusses how they work and provides some nice simple examples. The focus of the book is not reinforcement learning. The focus is on how to use Ray to scale up your workload. Which is why I am trying to go through the book with a newer version of Ray.

It seems like with the changes that have been made to Ray RLlib, the issue I am facing is not an easy to get working. Is that correct?

---

<div class="post-metadata">

**Author:** ![NICK\_SUPER](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/nick_super/32/8398_2.png) [@NICK\_SUPER](https://discuss.ray.io/u/NICK_SUPER)\
**Post date:** [December 15, 2025, 7:11am UTC](https://discuss.ray.io/t/attribute-error-when-using-custom-gymnasium-environment/21782/6 "2025-12-15T07:11:20Z")

</div>

这个问题我貌似搞定了，你是不是也在看那本封面有个大蝌蚪的 Ray分布式机器学习 第四章那里，我是ray 2.47.1，看源码猜着将 ray/rllib/algorithms/algorithm.py 文件的 大概第 3978行的 “if “EpisodeReplayBuffer” in config[“replay\_buffer\_config”][“type”]: ”改成 “if (isinstance(config[“replay\_buffer\_config”][“type”], str)) and “EpisodeReplayBuffer” in config[“replay\_buffer\_config”][“type”] :” 先

---

<div class="post-metadata">

**Author:** ![NICK\_SUPER](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/nick_super/32/8398_2.png) [@NICK\_SUPER](https://discuss.ray.io/u/NICK_SUPER)\
**Post date:** [December 15, 2025, 7:13am UTC](https://discuss.ray.io/t/attribute-error-when-using-custom-gymnasium-environment/21782/8 "2025-12-15T07:13:22Z")

</div>

然后我的完整训练脚本在这里：

#%%

import ray

print(f"Ray version: {ray.\__version_\_}")

import gymnasium as gym

from gymnasium.spaces import Discrete

import os

class Environment:

def \__init_\_(self, \*args, \*\*kwargs):

self.seeker, self.goal = (0, 0), (4, 4)

self.info = {‘seeker’: self.seeker, ‘goal’: self.goal}

self.action\_space = Discrete(4)

self.observation\_space = Discrete(5\*5)

# self.single\_action\_space = Discrete(4)

# self.single\_observation\_space = Discrete(5\*5)

def reset(self, seed, options): # seed, options

“”“Reset seeker position and return observations.”“”

self.seeker = (0, 0)

resetted\_infos = {‘seeker’:self.seeker} # resetted\_infos 是为了兼容加的

return self.get\_observation(), resetted\_infos # resetted\_infos 是为了兼容加的

def get\_observation(self):

“”“Encode the seeker position as integer”“”

return 5 \* self.seeker[0] + self.seeker[1]

def get\_reward(self):

“”“Reward finding the goal”“”

return 1 if self.seeker == self.goal else 0

def is\_done(self):

“”“We’re done if we found the goal”“”

return self.seeker == self.goal

def step(self, action):

“”“Take a step in a direction and return all available information.”“”

if action == 0: # move down

self.seeker = (min(self.seeker[0] + 1, 4), self.seeker[1])

elif action == 1: # move left

self.seeker = (self.seeker[0], max(self.seeker[1] - 1, 0))

elif action == 2: # move up

self.seeker = (max(self.seeker[0] - 1, 0), self.seeker[1])

elif action == 3: # move right

self.seeker = (self.seeker[0], min(self.seeker[1] + 1, 4))

else:

raise ValueError(“Invalid action”)

obs = self.get\_observation()

rew = self.get\_reward()

done = self.is\_done()

truncated = False # truncated 是为了兼容加的

return obs, rew, done, truncated, self.info # truncated 是为了兼容加的

def render(self, \*args, \*\*kwargs):

“”"We override this method here so clear the output in Jupyter notebooks.

The previous implementation works well in the terminal, but does not clear

the screen in interactive environments.

“”"

os.system(‘cls’ if os.name == ‘nt’ else ‘clear’)

try:

from IPython.display import clear\_output

clear\_output(wait=True)

except Exception:

pass

grid = [['| ’ for \_ in range(5)] + [“|\n”] for \_ in range(5)]

grid[self.goal[0]][self.goal[1]] = ‘|G’

grid[self.seeker[0]][self.seeker[1]] = ‘|S’

print(‘’.join([‘’.join(grid\_row) for grid\_row in grid]))

# maze\_gym\_env\_nick.py

class GymEnvironment(Environment, gym.Env):

def \__init_\_(self, \*args, \*\*kwargs):

super().\__init_\_(\*args, \*\*kwargs)

#%%

ray.shutdown()

from ray.rllib.evaluation.rollout\_worker import RolloutWorker

from ray.rllib.algorithms.dqn import DQNConfig

# from maze\_gym\_env\_nick import GymEnvironment

# #----------------------------废弃----------

# # conflg = DQNConfig().environment(“maze\_gym\_env\_nick.GymEnvironment”).rollouts(num\_rollout\_workers=2)

# #-----------------------------------------

# config = DQNConfig().environment(GymEnvironment).env\_runners(num\_env\_runners=2)

# # config = config.api\_stack(

# # enable\_rl\_module\_and\_learner=False,

# # enable\_env\_runner\_and\_connector\_v2=False

# # )

# #----------------------------------------

# algo = config.build()

# # from pprint import pprint

# # for \_ in range(4):

# # pprint(algo.train())

from ray.tune.logger import pretty\_print

# config = (DQNConfig().environment(GymEnvironment)

# .env\_runners(num\_env\_runners=2, create\_env\_on\_local\_worker=True))

config = (

DQNConfig()

.environment(GymEnvironment)

.framework(“torch”)

# 禁用新 API 栈，使用旧的稳定版本

.api\_stack(

enable\_rl\_module\_and\_learner=False,

enable\_env\_runner\_and\_connector\_v2=False

).

env\_runners(env\_runner\_cls = RolloutWorker)

.training(

replay\_buffer\_config={“type”: “ReplayBuffer”,“capacity”: 1000},

gamma=0.99,

lr=0.001

)

# .resources(num\_gpus=0)

# .env\_runners(num\_env\_runners=1)

# .debugging(log\_level=“WARN”)

)

pretty\_print(config.to\_dict())

algo = config.build()

for i in range(10):

result = algo.train()

print(pretty\_print(result))

---

<div class="post-metadata">

**Author:** ![NICK\_SUPER](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/nick_super/32/8398_2.png) [@NICK\_SUPER](https://discuss.ray.io/u/NICK_SUPER)\
**Post date:** [December 15, 2025, 7:15am UTC](https://discuss.ray.io/t/attribute-error-when-using-custom-gymnasium-environment/21782/9 "2025-12-15T07:15:25Z")

</div>

然后好像电脑就会响一会出一些打印的东西了