How to clean up in Gym env?

Hey,

this is maybe an amateur question, but I am wondering on how to properly clean up resources from a Gym env. According to the OpenAI docs, there is a close() method that you can implement, but it never gets called when I press CTRL-C. I also tried the much uglier solution with a SIGINT handler or atexit, but no luck either (at least not in the worker process that opens the resources).

To simplify, the training is started with:

register_env("my_env", MyEnv)
tune.run(run_or_experiment=tune.experiment.convert_to_experiment_list({
   experiment_id: {
      "run": "APEX"
      "env": "my_env"
   }
})[0])

Thanks for your help :slight_smile:

@sven1977 @michaelzhiluo @rliaw any ideas here?

@nschrader To check if SIGINT can clean up resources, try to fork() a bunch of processes running a gym environment and kill some processes while observing total RAM (it should go down).