# Does ray support multi GPU inference with TensorRT?

**URL:** <https://discuss.ray.io/t/does-ray-support-multi-gpu-inference-with-tensorrt/4583>\
**Category:** Ray Serve\
**Created:** [January 5, 2022, 10:53am UTC](https://discuss.ray.io/t/does-ray-support-multi-gpu-inference-with-tensorrt/4583 "2022-01-05T10:53:19Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![jinfagang](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/jinfagang/32/1947_2.png) [@jinfagang](https://discuss.ray.io/u/jinfagang)\
**Post date:** [January 5, 2022, 10:53am UTC](https://discuss.ray.io/t/does-ray-support-multi-gpu-inference-with-tensorrt/4583/1 "2022-01-05T10:53:19Z")

</div>

Does ray support multi GPU inference with TensorRT?

---

<div class="post-metadata">

**Author:** ![simon-mo](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ray.io/simon-mo/32/9_2.png) [@simon-mo](https://discuss.ray.io/u/simon-mo)\
**Post date:** [January 5, 2022, 6:31pm UTC](https://discuss.ray.io/t/does-ray-support-multi-gpu-inference-with-tensorrt/4583/2 "2022-01-05T18:31:46Z")

</div>

Hi @jinfagang, yes! You can just use TensorRT Python API in Ray Serve like local script because Ray Serve support arbitrary Python code. In order to use multiple GPU make sure to allocate GPUs via `@serve.deployment(ray_actor_options={"GPU": 2})` syntax.