RuntimeError: CUDA error: invalid device ordinal when have multiple ray deployment on the same GPU

[Andcircle]

For some reason I can’t edit or delete this one any more, so I created a new one when serve multiple models on a multi-gpu cluster, got error RuntimeError: CUDA error: invalid device ordinal