AttributeError: 'EnvContext' object has no attribute is_directed

RLlib
1

The context for this post is the same as this one: https://discuss.ray.io/t/registering-custom-env-that-passes-an-argument-in-1-13-0/8783. The code has been modified to look like the following:

def env_creator(env_config):
    return ind_set(env_config)

tune.register_env("myenv", env_creator)

G = nx.dodecahedral_graph()

if __name__ == "__main__":
    parser = argparse.ArgumentParser()
    parser.add_argument("--num-workers", default=6, type=int)
    parser.add_argument("--training-iteration", default=100, type=int) # change depending on graph size
    parser.add_argument("--ray-num-cpus", default=7, type=int)
    args = parser.parse_args()
    ray.init(num_cpus=args.ray_num_cpus)

    ModelCatalog.register_custom_model("dense_model", DenseModel)

    tune.run(
        "contrib/AlphaZero",
        stop={"training_iteration": args.training_iteration},
        max_failures=0,
        config={
            "env": "myenv",
            "env_config": {"graph": G},
            "num_workers": args.num_workers,
            "rollout_fragment_length": 10,
            "train_batch_size": 50,
            "sgd_minibatch_size": 8,
            "lr": 1e-4,
            "num_sgd_iter": 1,
            "mcts_config": {
                "puct_coefficient": 1.5,
                "num_simulations": 5,
                "temperature": 1.0,
                "dirichlet_epsilon": 0.20,
                "dirichlet_noise": 0.03,
                "argmax_tree_policy": False,
                "add_dirichlet_noise": True,
            },
            "ranked_rewards": {
                "enable": True,
            },
            "model": {
                "custom_model": "dense_model",
            },
        },
    )

Now, I get the error

(RolloutWorker pid=8796)   File "/home/IncompleteOmega/.local/lib/python3.8/site-packages/networkx/utils/decorators.py", line 86, in _not_implemented_for
(RolloutWorker pid=8796)     dval is None or dval == g.is_directed()
(RolloutWorker pid=8796) AttributeError: 'EnvContext' object has no attribute 'is_directed'

Note that I am using the networkx library (imported as nx). I suspect this is an issue with the way I have written the code and not the networkx library because of two reasons:

  1. My Networkx Module is updated to the newest version
  2. When I separately run G = nx.dodecahedral_graph() and then ask G.is_directed(), the result is the expected False ā€“ so this is functional in the correct way.

It would be great if someone could let me know how I can modify my code to get around this.

2

Hi @IncompleteOmega,

Just a quick check on this line,

return ind_set(env_config)

The env_config is an EnvContext object and is a subclass for dict.

Somewhere in ind_set you are pulling out the graph right?

graph_variable =env_config["graph"]

Alternatively if you do not want to pass in the EnvContext object you could call ind_set like this.

return ind_set(env_config["graph"])
3

Yes in the init function of ind_set, an argument taken as input is graph.

4

I believe it might be some sort of type error. When I changed the syntax to return ind_set(env_config["graph"]), I get a type error with the environment (the environment is fully functional though). The graph is supposed to be of the type nx.Graph(). Is there an issue with assigning "env_config": {"graph": G} because of this? Is it interpreting G as something else/can it not understand the nx.Graph() type?

5

@IncompleteOmega,

Hard to know what is going on without the full error.

6

The exact error for when I edit my code to use your version is the following:

ValueError: The observation collected from env.reset() was not  contained within your env's observation space. Its possible that There was a type mismatch, or that one of the sub-observations  was out of bounds: 

 reset_obs: None

 env.observation_space: Discrete(20)

 reset_obs's dtype: <class 'NoneType'>

 env.observation_space's dtype: int64

The observation space of my environment is a discrete object with size as the number of vertices in the graph. For the dodecahedral_graph I am using, this is 20. This is because the observation of a state is the vertices that are currently selected.

7

That error is saying that you either did not return a value in reset or the value you returned is None.

8

Yeah, Iā€™m setting self.state = None in the reset function. Oops. Let me fix this issue. Thanks for your help.

1 Like