Batching when using non python client

Ray Libraries (Data, Train, Tune, Serve) Ray Serve
1

Hi guys,

I am using Ray Serve to deploy my DL models and I am wondering if I can use batching when the requests are sent by the curl command for example. I don’t know how to return a list as a response.

Thanks in advance!

2

Hey @aamrani-dev, the batching is agnostic to how the requests are sent – if you send 5 queries simultaneously from 5 different curl commands, it’s likely that the requests will get batched in the backend.