# How do I enable remote inference for PPO?
**URL:** https://discuss.ray.io/t/how-do-i-enable-remote-inference-for-ppo/3025
**Category:** RLlib
**Created:** [July 26, 2021, 8:55pm UTC](https://discuss.ray.io/t/how-do-i-enable-remote-inference-for-ppo/3025 "2021-07-26T20:55:18Z")
**Posts on this page:** 4
**Page:** 1
### Author:  [@Dylan\_Kerler](https://discuss.ray.io/u/Dylan_Kerler)
#### Post date: [July 26, 2021, 8:55pm UTC](https://discuss.ray.io/t/how-do-i-enable-remote-inference-for-ppo/3025/1 "2021-07-26T20:55:18Z")
I want to step n envs in parallel on 1 node, collect n observations, send n observations off to another node with 12 GPUs that hold the policy weights, compute n actions on those 12 GPUs, then send n actions back to the original node. Then repeat from step one again.
Is this possible with rllib?
---
### Author:  [@sven1977](https://discuss.ray.io/u/sven1977)
#### Post date: [July 28, 2021, 3:14pm UTC](https://discuss.ray.io/t/how-do-i-enable-remote-inference-for-ppo/3025/2 "2021-07-28T15:14:49Z")
Hey @Dylan_Kerler , so far, none of the RLlib algos does this kind of “remote” inference (calculating actions on a node different from the one holding the envs). I’m guessing you’d like to have something that resembles DeepMind’s “SEED” architecture?
But there are some settings, which could help you achieve this:
- Assuming you have your 12 GPU machine (+ some small number of CPUs) and one or more CPU-only nodes (on which you would like to run your envs).
- Set `num_workers=0`, you’ll only have a single learner that also does the sampling.
- Set `remote_worker_envs=True`. This will make each individual env a ray actor. All n envs are then stepped in parallel.
- You would then also have to override the `default_resource_request` method in your Trainer to make sure the env CPUs are not required to be on the same node as the GPUs.
You can look at how IMPALA does this in `ray.rllib.agents.impala.impala.py`. Something like this may work:
```auto
class OverrideDefaultResourceRequest:
@classmethod
@override(Trainable)
def default_resource_request(cls, config):
cf = dict(cls._default_config, **config)
# Return PlacementGroupFactory containing all needed resources
# (already properly defined as device bundles).
return PlacementGroupFactory(
bundles=[{
"CPU": cf["num_cpus_for_driver"],
"GPU": cf["num_gpus"]
}, {
# Different bundle (node) for your n "remote" envs (set remote_worker_envs=True).
"CPU": cf["num_envs_per_worker"]
}],
strategy=config.get("placement_strategy", "PACK"))
MyTrainer = add_mixins(PPOTrainer, [OverrideDefaultResourceRequest])
```
I haven’t tried this, but I can spend some time on a simple example script that would demonstrate such a setup. …
---
### Author:  [@sven1977](https://discuss.ray.io/u/sven1977)
#### Post date: [July 28, 2021, 5:00pm UTC](https://discuss.ray.io/t/how-do-i-enable-remote-inference-for-ppo/3025/3 "2021-07-28T17:00:03Z")
Here is an example script that demonstrates how to do this:
>
>
> Add example script for how to do n remote envs with inference happening on "main…" (possibly GPU) node.
>
> Also see this discussion here:
> https://discuss.ray.io/t/how-do-i-enable-remote-inference-for-ppo/3025/2
>
>
>
> \## Why are these changes needed?
>
>
>
> \## Related issue number
>
>
>
> \## Checks
>
> \- \[\] I've run \`scripts/format.sh\` to lint the changes in this PR.
> \- \[\] I've included any doc changes needed for https://docs.ray.io/en/master/.
> \- \[\] I've made sure the tests are passing. Note that there might be a few flaky tests, see the recent failures at https://flakey-tests.ray.io/
> \- Testing Strategy
> - \[\] Unit tests
> - \[\] Release tests
> - \[\] This PR is not tested :(
---
### Author:  [@sven1977](https://discuss.ray.io/u/sven1977)
#### Post date: [August 19, 2021, 10:15am UTC](https://discuss.ray.io/t/how-do-i-enable-remote-inference-for-ppo/3025/4 "2021-08-19T10:15:14Z")
This has been merged into master. Sorry for the delay.