Do I have to change "input": "sampler" config when working with ExternalEnv API?

RLlib
1

Hello,
Do I always have to change the config settings from “input”: “sampler” to “input”: [callable ray.rllib.offline.InputReader] when working with ExternalEnv API?

2

Yes, because your env is “external”, meaning all data from the env comes in via tcp (from the client).
If you set “input” to sampler, the data from the external env would not reach the learner (it would still get its data from “regular” RolloutWorkers working in local envs).

1 Like
3

Thanks for your reply @sven1977!
Does this imply that in the custom external env I must access the rollout worker (e.g. self.rollout_worker = ioctx.worker, see PolicyServerInput) and also sample from it like in class _LocalInferenceThread (see policy_client.py) to send an input object to InputReader?

4

Ah, I see. Yeah, you have to set up a dummy RandomEnv on the server side. Sorry about that inconvenience, this is a known cause for confusion.

You don’t really have to do anything else, other then the config changes shown in these examples here:

On the client side (where you external env sits), you need to send data to the server via the external env API and on the server side (learner), this data will be automatically processed by the rollout worker (the sampler (with the dummy env) is not used here).

5

I think what you’re suggesting @sven1977 doesn’t really work for my case since my client side isn’t in Python. I understand that in the cartpole_client.py example the standard gym.Env will be automatically wrapped in the ExternalEnv interface and then everything will be handled by PolicyClient and PolicyServerInput.
Thus, I thought I need to employ an ExternalEnv on my server side where I send data to (a rollout worker to process data and to generate experiences).
Is this then a case for “input” equals to “sampler”? Or is a possible practice the one I tried to explain in my first reply? Sorry, but my confusion still exists.