MultiAgents type actions/observation space defined in environement

RLlib
5

Hi @sven1977

Unfortunately, the on_sub_environment_created(self,*,worker: "RolloutWorker",sub_environment: EnvType,env_context: EnvContext,**kwargs, ) -> None:
doesn’t give me a reference to the trainer, but I think I could make it work by combining it with the on_trainer_init callback
Howerver, thoses are not available in the 1.10 or the 1.11 preview on pip. I could install ray from master but i’d like to stay on official releases.

Here is an example of what I’d like to achieve:

I took the same idea as the MultiAgentTrafficEnv from the docs

However, in the docs, the obs and action spaces are defined in the multiagent policies configs, in my case, I want to define them in the environement. (Because I get them from the simulation env)

In Ray 1.10, only a single and obs space could be defined in the env, making it impossible for multiple agent types to be defined within the env, and policies config beforehand was required

In ray 1.11 preview, it should be possible to define a dict, mapping per-agent spaces in this way:

self.observation_spaces = spaces.Dict({
    "car_0":  CAR_OBS_SPACE,
    "car_1":  CAR_OBS_SPACE,   
    "traffic_light_0": TRAFFIC_LIGHT_OBS_SPACE,
     ...
)}

This functionality has been implemented in the following commit: [RLlib] [MultiAgentEnv Refactor #2] Change space types for `BaseEnvs`… · ray-project/ray@39f8072 · GitHub
Unfortunately it is not working as expected