TypeError: 'EnvContext' object cannot be interpreted as an integer

RLlib
1

I’ve search the forum and saw previous posts on this issue. I tried the solutions suggested in those posts, however they didn’t work out.

Screen Shot 2021-11-10 at 12.52.44 AM
Screen Shot 2021-11-10 at 12.52.44 AM1368×261 66.6 KB

To replicate my error:

from ray import tune
import gym, ray
from ray.rllib.agents import ppo
from envs.cybersec import CybersecEnv

Register , how your env should be constructed (always with 5, or you can take values from the config EnvContext object):

tune.register_env(“my_env”, lambda config: CybersecEnv(2))

ray.init()
trainer = ppo.PPOTrainer(env=CybersecEnv, config={
“env_config”: {
“num_agents”: 2, # ← env_config will be passed into your Env’s c’tor as config
},
})

Part of the implementation of my customized environment is as follows:

class CybersecEnv(gym.Env):
    def __init__(self, num_agents=2, num_adversaries=1, num_edges=5,
                 num_vertices=4, num_bins=3, adversary_types=[], eta=1.0,
                 poss_adv_types=['w', 'p'], graph=None, p_obs=0.8, k=1,
                 def_budget=5):
        self.re_init(
            num_agents, num_adversaries, num_edges, num_vertices, num_bins,
            adversary_types, eta, poss_adv_types, graph, p_obs, k, def_budget
        )
        # TODO:
        #   introduce defender budget for adjusting perceived target values
        #   implement visualization for inspecting resulting policies
        #   double check that the correct information is being conveyed to the specific attackers (w vs p)
    
    def re_init(self, num_agents, num_adversaries, num_edges, num_vertices,
                num_bins, adversary_types, eta, poss_adv_types, graph, p_obs,
                k, def_budget):
        self.num_agents = num_agents
        self.num_adversaries = num_adversaries 
        self.num_edges = num_edges
        self.num_vertices = num_vertices
        self.poss_adv_types = poss_adv_types
        self.eta = eta 
        self.p_obs = p_obs
        self.k = k
        self.n = num_agents
        self.def_budget = def_budget
        self.remaining_budget = def_budget
        if not adversary_types:
            self.adversary_types = np.random.choice(
                np.asarray(poss_adv_types), 
                size=num_adversaries,
                p=np.asarray([eta, 1-eta])
            )
        else:
            self.adversary_types = adversary_types
        self.num_bins = num_bins # number of bins for discretizing target values
        self.action_space = self.init_action_space()
        self.observation_space = self.init_obs_space()
        self.obs_shapes = [self.observation_space[i].shape for i in range(self.num_agents)]
        self.graph, self.vals = self.init_graph(graph)
        print(self.graph)
        self.entry_pts = self.get_entry_points()
        self.reset()

        # rendering 
        self.locs = []
        overlapping = True  # TODO implement additional overlapping test
        overlap_diff = 0.5  # TODO test overlapping 
        while len(self.locs) < self.num_vertices:
            xpos = np.random.choice(np.arange(-4, 4))
            ypos = np.random.choice(np.arange(-4, 4))
            if (xpos, ypos) not in self.locs:
                self.locs.append((xpos, ypos))
        self.shared_viewer = True 
        if self.shared_viewer:
            self.viewers = [None]
        self.reset_render()

Could you please help see why this error occurs? I feel not quite appropriate to share the full implementation in the forum, are there any other more private communication channel? Thanks!

2

Hi @Yinuo_Du ,

and welcome to the ray community. Just a suggestion after reading @sven1977 answer: If you want to use an env_config for your environment, you might need to define this in your c’tor of it like this:

class MyEnv(gym.Env):
    def __init__(self, env_config=None): 
         super(MyEnv, self).__init__()
         self.config = env_config or {}
         self.var1 = self.config.get("var1", 0.0)
         ...

Hope I could help here.
Simon

3

Thanks for your answer! Do I need to do the same in the re_init() function?

4

I can only guess that you mean the reset() method and no only in the __init__().

1 Like