# Running PolicyServer code asynchronously in background

**URL:** <https://discuss.ray.io/t/running-policyserver-code-asynchronously-in-background/3641>\
**Category:** RLlib\
**Created:** [September 26, 2021, 5:02pm UTC](https://discuss.ray.io/t/running-policyserver-code-asynchronously-in-background/3641 "2021-09-26T17:02:30Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![import-antigravity](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/import-antigravity/32/562_2.png) [@import-antigravity](https://discuss.ray.io/u/import-antigravity)\
**Post date:** [September 26, 2021, 5:02pm UTC](https://discuss.ray.io/t/running-policyserver-code-asynchronously-in-background/3641/1 "2021-09-26T17:02:30Z")

</div>

I’m trying to implement something similar to the examples mentioned [here](https://discuss.ray.io/t/trying-to-set-up-external-rl-environment-and-having-trouble/3265/3) with a PolicyClient and PolicyServer but I’d like to start the client programmatically in the background. I don’t think I can just put them in separate scripts and run them completely separately because the server needs information about the state-action space at instantiation time.

I’ve tried doing something like this:

```python
@ray.remote
class RLServer:
    def __init__ (self, args):
        # set things up
        self.trainer = SACTrainer(...)

    async def train_loop(self):
        for _ in range(self.stop_iters):
            results = self.trainer.train()
            ...

```

```python
# Main code
class Experiment:
    def setup(self):
        ...
        self.server = RLServer(...)

    def run(self):
        self.server.train_loop.remote()
        # Start PolicyClient...

```

Can anyone help me figure out what to do? If I just do `.remote()` then the server never starts, if I do `ray.get()` then the server starts but it blocks.

---

<div class="post-metadata">

**Author:** ![sven1977](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/sven1977/32/53_2.png) [@sven1977](https://discuss.ray.io/u/sven1977)\
**Post date:** [September 27, 2021, 7:21am UTC](https://discuss.ray.io/t/running-policyserver-code-asynchronously-in-background/3641/2 "2021-09-27T07:21:01Z")

</div>

Hey @import-antigravity , thanks for the question. I think it would be much easier to use the PolicyClient|Server classes and APIs (+ examples). You can give the server space hints via your (SAC) config, e.g.:

```auto
observation_space: Box(-1.0, 1.0, (3, ), float32)
action_space: Discrete(5)

```

The Server - by default - won’t create an env, but will know via your space-hints, which policy to build (and learn).

---

<div class="post-metadata">

**Author:** ![import-antigravity](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/import-antigravity/32/562_2.png) [@import-antigravity](https://discuss.ray.io/u/import-antigravity)\
**Post date:** [September 27, 2021, 1:12pm UTC](https://discuss.ray.io/t/running-policyserver-code-asynchronously-in-background/3641/3 "2021-09-27T13:12:11Z")

</div>

That’s what I’m doing

---

<div class="post-metadata">

**Author:** ![rusu24edward](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/rusu24edward/32/333_2.png) [@rusu24edward](https://discuss.ray.io/u/rusu24edward)\
**Post date:** [September 28, 2021, 5:54pm UTC](https://discuss.ray.io/t/running-policyserver-code-asynchronously-in-background/3641/4 "2021-09-28T17:54:42Z")

</div>

@import-antigravity Have you tried using the API that Sven suggested? You can find an example here:

> <https://github.com/ray-project/ray/blob/master/rllib/examples/serving/cartpole_server.py>

> <https://github.com/ray-project/ray/blob/master/rllib/examples/serving/cartpole_client.py>