How can I assign different GPU for different replicas in Ray Serve?

Ray Libraries (Data, Train, Tune, Serve) Ray Serve
1

Hi, I have 2 GPUs and want to start 6 instances (num_replicas=6). How can I assign the first 3 instances to GPU-1 and the remaining 3 instances to GPU-2 ?.

2

Hi @Anh_Minh_Nguyen_Quan , I checked GPU Support — Ray 3.0.0.dev0 and didn’t find anything about Ray supporting running tasks on specific GPUs.

However, can you try the following?
@serve.deployment(ray_actor_options={"num_gpus": 0.33})
I think this should have the effect that only 3 replicas will use each GPU.