How to increase ray performance for cpu and io bound operations in a task

Ray Core
1

Hello Team,
I need to get data by calling some rest api and then need to perform some operations on it. Api call sometime takes 11 seconds and operations are cpu bound those takes few milliseconds.

Currently I kept everything in task but thinking to split getting data into async actor and computation in task.

I am running this task with single cpu and 1 gb ram.
Suppose I have to run 1000 time this task concurrently then how should I go .

Does this is correct approach or I am making some mistakes.

2

are you trying to run tasks concurrently or in a parallel manner? if data remains unchanged have to tried adding data to ray’s object store?

3

I am putting objects in ray object store only.

4

thanks, from what I know ray spawns processes to run tasks in a parallel and distributed manner. if you have a cluster.

Sharing a code snippet that may help

#spawn 1000 tasks of your_func
tasks = [your_func for _ in range(1000)]
#get results of all 1000 tasks
print(ray.get(tasks))

5

Yeah this is the right way to go. You can have a async actor which calls the API and invoke a task based on the result; example;

class APIReadActor:
    async def run(self):
        while True:
            data = call_api()
            await ray.get(cpu_task.remote(data))
            await asyncio.sleep(0) # This is a hack to run while loop in an actor while it can still process other requests. Note that the actor is always single threaded, so if you run while loop in one of its tasks, it cannot process other tasks originally.
6

Another approach can be to allocate fractional CPUs to tasks, for example:

@ray.remote(num_cpus=0.1)
def io_task():
pass

This is simpler than the actor approach, but has some limitations in that you cannot reduce the number of CPUs indefinitely small, since each task still needs a worker process.

7

Thank you @sangcho . Will publish the result of this… I will be trying making this change and see how this works.

8

thank you for reply. this still consume some sort of cpu…

9

One quick question.
Does each async actor will be executed in separate event loop?..
If yes then i can create separate actor for each i/o operations…

10

async actor shares the same event loop. Btw, spending some cpus are expected because it still needs to run some code within the actor. Are you saying the cpu consumption is high?