# Launching subprocesses inside a ray.remote is 10x slower

**URL:** <https://discuss.ray.io/t/launching-subprocesses-inside-a-ray-remote-is-10x-slower/13719>\
**Category:** Ray Core\
**Created:** [February 14, 2024, 2:07am UTC](https://discuss.ray.io/t/launching-subprocesses-inside-a-ray-remote-is-10x-slower/13719 "2024-02-14T02:07:44Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![frib](https://avatars.discourse-cdn.com/v4/letter/f/90ced4/32.png) [@frib](https://discuss.ray.io/u/frib)\
**Post date:** [February 14, 2024, 2:07am UTC](https://discuss.ray.io/t/launching-subprocesses-inside-a-ray-remote-is-10x-slower/13719/1 "2024-02-14T02:07:44Z")

</div>

Launching subprocesses using asyncio.create\_subprocess\_exec inside a ray.remote is 10x slower. For example:

```python
import asyncio
import ray
from tqdm.asyncio import tqdm_asyncio

async def arun():
    await tqdm_asyncio.gather(*[
        asyncio.create_subprocess_exec("bash", "-c", "sleep 0.0001")
        for _ in range(3000)
    ])

@ray.remote
def run():
    asyncio.run(arun())

ray.init()

asyncio.run(arun()) # 3s, 1k it/s
ray.get(run.remote()) # 30s, 100 it/s (10x slower)

```

It’s even worse when the task is more heavy (e.g. you get a 30x when cating a file to dev/null)  
I’m using ray-2.9.2 and python 3.10.11 in Ubuntu 22.04

The use case: I want to train a model to write correct code, and during eval I generate many programs and run them against test cases.

I don’t understand why it slows down, and I would really appreciate an explanation of why this happens and how to fix it!