Training ray.rllib algorithm with vectorized environments

sai_lalith_Polawar · February 7, 2022, 11:56am

I have created a custom vectorized environment using static method as mentioned in https://stackoverflow.com/q/70988560/18120335 and the question is how to use the return of that static method to train an algorithm. Is there any option to pass our own vec env to algorithms in ray.rllib? If this is not used to train any algorithm then what is this static method used for?

avnishn · February 8, 2022, 1:22am

If your environment is a gym environment, then you actually shouldn’t need to manually need to vectorize your gym environment. RLlib experiments automatically vectorize the gym environment that is passed.

To see an example of using a custom gym environment with our launchers, see this example:

github.com

ray-project/ray/blob/ee1711fe418e44818ff791b266cd9dea42a65957/rllib/examples/custom_env.py#L155

      
        
                    return torch.reshape(self.torch_sub_model.value_function(), [-1])
            
            

            
if __name__ == "__main__":
                args = parser.parse_args()
                print(f"Running with following CLI options: {args}")
            
            
    ray.init(local_mode=args.local_mode)
            
            
    # Can also register the env creator function explicitly with:
                # register_env("corridor", lambda config: SimpleCorridor(config))
                ModelCatalog.register_custom_model(
                    "my_model", TorchCustomModel if args.framework == "torch" else CustomModel
                )
            
            
    config = {
                    "env": SimpleCorridor,  # or "corridor" if registered above
                    "env_config": {
                        "corridor_length": 5,
                    },
                    # Use GPUs iff `RLLIB_NUM_GPUS` env var set to > 0.

:

Topic		Replies	Views
Only use Ray to vectorize environment RLlib	4	287	July 15, 2021
Custom VectorEnv for efficient parallel environment evaluation RLlib	8	590	May 27, 2021
Custom Gym Environment NaN RLlib	0	197	June 16, 2023
Custom simulator with as RLlib environment RLlib	1	375	December 17, 2020
Issue regarding custom gym environment with Ray Rlib RLlib	1	273	October 23, 2023

Training ray.rllib algorithm with vectorized environments

Related Topics