I’m just starting out with RLLib and I was wondering what is the best way to interface an RL agent with a client’s sim (agents gets observations from sim then drives some control variables).
The problem is that the sim is not in a ‘gym format’. My current idea is to initialize the sim inside a custom gym env, during each iteration of the gym (and thereby the sim) feed the state variables into the agent, then feed the agent’s actions into the sim (repeat and hope for the best) .
To wrap your simulator inside a gym env is the preferable and standard solution. Sometimes you need to break out of ray for that. If so: I have answered a related question here.
Hey @arturn so this a really rough idea of how I want wrap the sim with a custom gym env. Some background, the sim that I’m working with has callbacks to grab state variables, input control variable and iterate with a time step (makes my life a lot easier)
class WrapperEnv(WrapperEnv):
def __init__(self,config):
self.sim_bot = SimBot()
self.observation_space = vector_space
self.action_space = vector_space
self.time_step = 0.020
def step(self):
_backend_method_for_grabbing_current_sim_state_variables()
_backend_mehod_for_sim_controls()
_calcuate_reward()
_update_sim()
_increment_sim()
def reset(self):
_reset_state_vars()
_reset_sim()
def render(self):
# do I need to render the bot actions ??
# can it just be a bunch of print outs
You need to implement render() if you tell RLlib to do so in your config. Without doing so, "render_env": False is set by default. You will be fine as long as you adhere to the gym.Env interface. It’s absolutely ok it your simulator needs some time.
You rough idea looks like a good rough idea
