ModuleNotFoundError: No module named 'ray.serve.llm'

Ray Serve
1

Trying to just deploy LLM using ray server documentation using below link we are getting the llm module error. i am having ray serve below version
pip show ray
WARNING: Ignoring invalid distribution -orch (/usr/local/lib/python3.9/site-packages)
Name: ray
Version: 2.42.1

when i try to run the code from the link “Serving LLMs — Ray 3.0.0.dev0” getting this module error

2

Hi @RajaSekhar_Kavuri , the llm apis are available in ray >= 2.43. It seems like you are on 2.42.1. We changed the import paths and some namings a bit from 2.43 to master (to be released in 2.44). so make sure you use the right doc version when testing out.