Does ray-llm support only CPU?

saranya_gouthem October 25, 2023, 4:35pm 1

Does ray-llm model scaling config support only CPU?
or GPU is a must to run Aviary model ?

Topic		Replies	Views
Serving LLM with multiple gpus LLMs/Generative AI/Aviary	0	104	July 3, 2024
LLM model loading LLMs/Generative AI/Aviary	0	30	July 29, 2024
Question about the model yaml config `accelerator_type_a100` LLMs/Generative AI/Aviary	1	430	February 2, 2024
Memory Requirements for distributing LLM LLMs/Generative AI/Aviary	3	556	August 31, 2023
Best practices to run multiple models in multiple GPUs in RayLLM LLMs/Generative AI/Aviary	0	644	February 8, 2024