# How can i run two PPO in parallel or in sequence?

**URL:** <https://discuss.ray.io/t/how-can-i-run-two-ppo-in-parallel-or-in-sequence/7835>\
**Category:** RLlib\
**Created:** [October 10, 2022, 9:34am UTC](https://discuss.ray.io/t/how-can-i-run-two-ppo-in-parallel-or-in-sequence/7835 "2022-10-10T09:34:36Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![imxuemei](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/imxuemei/32/3295_2.png) [@imxuemei](https://discuss.ray.io/u/imxuemei)\
**Post date:** [October 10, 2022, 9:34am UTC](https://discuss.ray.io/t/how-can-i-run-two-ppo-in-parallel-or-in-sequence/7835/1 "2022-10-10T09:34:36Z")

</div>

I want to test different custom models of PPO. Then, i hope to run two PPO in parallel or insequence. How can i do?  
One example code is as follows. But it ends with errors. a1.stop() or ray.shutdown() can not help.  
software versions:  
ray and rllib: 2.0.0  
python: 3.8  
pytorch: 1.10.2

```auto
import ray
from ray.rllib.algorithms.ppo import PPO

config = {
    "framework": "torch",
    "env": "CartPole-v0",
    "gamma": 0.0,
}
ray.init(include_dashboard=False)
a1 = PPO(config = config)
res1 = a1.train()
print("a1 res")
# a1.stop()
# ray.shutdown()

a2 = PPO(config = config)
res2 = a2.train()
print("a2 res")

```

Error:

> 2022-10-10 17:25:05,752 WARNING worker.py:1829 – The node with node id: 67b8ed4a54504877a25602a5aba40d4ad4ca1daff2fea7e9f7e6009b and address: 127.0.0.1 and node name: 127.0.0.1 has been marked dead because the detector has missed too many heartbeats from it. This can happen when a (1) raylet crashes unexpectedly (OOM, preempted node, etc.)  
> (2) raylet has lagging heartbeats due to slow network or busy workload.  
> Traceback (most recent call last):  
> File “E:\workspace-bupt\06-offload-sdag\0-code\test\_rllib.py”, line 16, in   
> a2 = PPO(config = config)  
> File “E:\ProgramData\Anaconda3\envs\rllib\lib\site-packages\ray\rllib\algorithms\algorithm.py”, line 308, in **init**  
> super(). **init** (config=config, logger\_creator=logger\_creator, \*\*kwargs)  
> File “E:\ProgramData\Anaconda3\envs\rllib\lib\site-packages\ray\tune\trainable\trainable.py”, line 157, in **init**  
> self.setup(copy.deepcopy(self.config))  
> File “E:\ProgramData\Anaconda3\envs\rllib\lib\site-packages\ray\rllib\algorithms\algorithm.py”, line 446, in setup  
> raise e  
> File “E:\ProgramData\Anaconda3\envs\rllib\lib\site-packages\ray\rllib\algorithms\algorithm.py”, line 418, in setup  
> self.workers = WorkerSet(  
> File “E:\ProgramData\Anaconda3\envs\rllib\lib\site-packages\ray\rllib\evaluation\worker\_set.py”, line 125, in **init**  
> self.add\_workers(  
> File “E:\ProgramData\Anaconda3\envs\rllib\lib\site-packages\ray\rllib\evaluation\worker\_set.py”, line 269, in add\_workers  
> self.foreach\_worker(lambda w: w.assert\_healthy())  
> File “E:\ProgramData\Anaconda3\envs\rllib\lib\site-packages\ray\rllib\evaluation\worker\_set.py”, line 391, in foreach\_worker  
> remote\_results = ray.get([w.apply.remote(func) for w in self.remote\_workers()])  
> File “E:\ProgramData\Anaconda3\envs\rllib\lib\site-packages\ray\_private\client\_mode\_hook.py”, line 105, in wrapper  
> return func(\*args, \*\*kwargs)  
> File “E:\ProgramData\Anaconda3\envs\rllib\lib\site-packages\ray\_private\worker.py”, line 2277, in get  
> raise value  
> ray.exceptions.RayActorError: The actor died unexpectedly before finishing this task.  
> class\_name: RolloutWorker  
> actor\_id: 1519d88f54c329d1a94af12201000000  
> pid: 15076  
> namespace: fd8c3f60-1ea0-4852-80f4-473d2d605a4d  
> ip: 127.0.0.1  
> The actor is dead because its owner has died. Owner Id: 01000000ffffffffffffffffffffffffffffffffffffffffffffffff Owner Ip address: 127.0.0.1 Owner worker exit type: SYSTEM\_ERROR Worker exit detail: Owner’s node has crashed.  
> The actor never ran - it was cancelled before it started running.  
> Process finished with exit code 1

---

<div class="post-metadata">

**Author:** ![Lars\_Simon\_Zehnder](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/lars_simon_zehnder/32/1185_2.png) [@Lars\_Simon\_Zehnder](https://discuss.ray.io/u/Lars_Simon_Zehnder)\
**Post date:** [October 10, 2022, 1:28pm UTC](https://discuss.ray.io/t/how-can-i-run-two-ppo-in-parallel-or-in-sequence/7835/2 "2022-10-10T13:28:43Z")

</div>

Hi @imxuemei and welcome to the forum. You might want to use `ray.tune` for this experiment. It offers you the possibility to run multiple trials with different configuration settings. See [this link](https://docs.ray.io/en/latest/tune/examples/pbt_ppo_example.html) for an example.

---

<div class="post-metadata">

**Author:** ![imxuemei](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/imxuemei/32/3295_2.png) [@imxuemei](https://discuss.ray.io/u/imxuemei)\
**Post date:** [October 10, 2022, 3:00pm UTC](https://discuss.ray.io/t/how-can-i-run-two-ppo-in-parallel-or-in-sequence/7835/3 "2022-10-10T15:00:44Z")

</div>

Thanks!  
I also run [rllib/examples/sb2rllib\_rllib\_example.py] for a test, but it still returns an error related to ray actor.  
I install ray 3.0.0.dev, and it can work well.