Ray Serve


Ray Serve LLM APIs Ray Serve has LLM APIs to provide an easy way to deploy and scale multiple LLM models with a unified API. It supports automatic scaling, multi-model deployment, OpenAI-compatible endpoints, and LoRA multiplexing. The engine-agnostic architecture works with frameworks like vLLM and SGLang, enabling efficient model serving across multiple nodes.
Topic Replies Views Activity
0 798 November 17, 2020
1 5 June 23, 2025
3 44 June 20, 2025
14 1604 June 19, 2025
2 30 June 16, 2025
3 24 June 11, 2025
5 66 June 11, 2025
1 20 June 10, 2025
3 54 June 3, 2025
2 32 March 25, 2025
1 26 May 20, 2025
7 74 May 20, 2025
0 19 May 20, 2025
7 145 May 19, 2025
1 13 May 19, 2025
2 621 May 16, 2025
2 33 May 9, 2025
0 19 May 8, 2025
2 26 May 6, 2025
1 28 May 6, 2025
4 217 April 24, 2025
0 16 April 24, 2025
5 61 April 21, 2025
5 60 April 17, 2025
2 1098 April 17, 2025
3 47 April 16, 2025
9 94 April 11, 2025
2 63 April 7, 2025
2 46 April 5, 2025
2 458 April 3, 2025