Registering Custom Env That Passes an Argument in 1.13.0

RLlib
1

I’m using Ray 1.13.0 because of the AlphaZero implementation (ray/rllib/contrib/alpha_zero). I have a custom gym environment ind_set with id “IndependentSet-v0”. The init function of the gym environment passes an argument “graph” as well as self. I’ve imported the necessary libraries, then created an environment creator:

def create_env(config):
    return ind_set(graph=config["graph"])

And then ran this large block of code:


if __name__ == "__main__":
    parser = argparse.ArgumentParser()
    parser.add_argument("--num-workers", default=6, type=int)
    parser.add_argument("--training-iteration", default=100, type=int) # change depending on graph size
    parser.add_argument("--ray-num-cpus", default=7, type=int)
    args = parser.parse_args()
    ray.init(num_cpus=args.ray_num_cpus)

    tune.register_env("my_env", create_env)
    ModelCatalog.register_custom_model("dense_model", DenseModel)

    tune.run(
        "contrib/AlphaZero",
        stop={"training_iteration": args.training_iteration},
        max_failures=0,
        config={
            "env": "my_env",
            "graph": nx.dodecahedral_graph(),
            "num_workers": args.num_workers,
            "rollout_fragment_length": 10,
            "train_batch_size": 50,
            "sgd_minibatch_size": 8,
            "lr": 1e-4,
            "num_sgd_iter": 1,
            "mcts_config": {
                "puct_coefficient": 1.5,
                "num_simulations": 5,
                "temperature": 1.0,
                "dirichlet_epsilon": 0.20,
                "dirichlet_noise": 0.03,
                "argmax_tree_policy": False,
                "add_dirichlet_noise": True,
            },
            "ranked_rewards": {
                "enable": True,
            },
            "model": {
                "custom_model": "dense_model",
            },
        },
    )

This code, when run, gives the error

(AlphaZeroTrainer pid=8366)   File "/usr/local/lib/python3.8/dist-packages/ray/util/ml_utils/dict.py", line 52, in deep_update
(AlphaZeroTrainer pid=8366)     raise Exception("Unknown config parameter `{}` ".format(k))
(AlphaZeroTrainer pid=8366) Exception: Unknown config parameter `graph`

I’ve tried multiple alternatives, not of which are working. Is there another way to register a custom env that passes an argument, which is compatible with the same tune.run() function format as I have in the code above?

2

Hi @IncompleteOmega,

There is a sub-dictionary for the environment config.

I think you want

config = {
... 
    "env_config" = { "graph": nx.dodecahedral_graph()}, 
... 
}