I developed a custom multi-agent hierarchical environment using the MultiAgentEnv class as a starting point.
It is based on a quite sizeable store of data(the state) incorporating different data formats. From there the agents pull their observations. According to the agents actions the data store/state has to be updated/modified with some demanding computations. These calculations also rely heavily on other python modules.
Has anyone a recommendation if this should be best used as external environment or should i put everything in the env class and let it be stepped by RLlib? I also wonder what’s the better option performance-wise. Does Ray accelerate only the learning part or also the env steps when using GPUs? Thanks in advance!