Interfacing an RL-LIB agent with a non gym env

Hey guys,

I’m just starting out with RLLib and I was wondering what is the best way to interface an RL agent with a client’s sim (agents gets observations from sim then drives some control variables).

The problem is that the sim is not in a ‘gym format’. My current idea is to initialize the sim inside a custom gym env, during each iteration of the gym (and thereby the sim) feed the state variables into the agent, then feed the agent’s actions into the sim (repeat and hope for the best) .

Sorry if this sounds stupid :sweat_smile:

Hi @Stale_neutrino ,

To wrap your simulator inside a gym env is the preferable and standard solution. Sometimes you need to break out of ray for that. If so: I have answered a related question here.

Is this enough to get you started?

1 Like

Hey @arturn thanks for the info ! I’ll try to come up with some pseudo code tomorrow and if it’s alright I’d like to get your input on it.

1 Like

Hey @arturn so this a really rough idea of how I want wrap the sim with a custom gym env. Some background, the sim that I’m working with has callbacks to grab state variables, input control variable and iterate with a time step (makes my life a lot easier)

class WrapperEnv(WrapperEnv):
    def __init__(self,config):
        self.sim_bot = SimBot()
        self.observation_space = vector_space
        self.action_space = vector_space
        self.time_step = 0.020
    def step(self):
        _backend_method_for_grabbing_current_sim_state_variables()
        _backend_mehod_for_sim_controls()
        _calcuate_reward()
        _update_sim()
        _increment_sim()
    def reset(self):
        _reset_state_vars()
        _reset_sim()
    def render(self):
        # do I need to render the bot actions ??
        # can it just be a bunch of print outs

hope this makes some sense…

Hi @Stale_neutrino ,

You need to implement render() if you tell RLlib to do so in your config. Without doing so, "render_env": False is set by default. You will be fine as long as you adhere to the gym.Env interface. It’s absolutely ok it your simulator needs some time.
You rough idea looks like a good rough idea :slightly_smiling_face:

Cheers

1 Like

@arturn

Cheers dude ! Thanks for all the help

1 Like