Question on Creating Backend with Serve

Ray Libraries (Data, Train, Tune, Serve) Ray Serve
1

I’m getting a very strange error when trying to run a very basic ray program.

import ray
from ray import serve
import time

ray.init()

# This will start Ray locally and start Serve on top of it.
serve.start()

def my_backend_func(request):
  return "hello"

serve.create_backend("my_backend", my_backend_func)

Running this gives me the following error: AttributeError: module 'ray.serve' has no attribute 'create_backend' .

If I store the object created by calling serve.start() in a variable, and use that to call .create_backend instead of serve it works. Every single test case, example, etc. does not do this so I was wondering what I might be doing wrong. I was able to recreate this issue on every Linux machine I tried it on, and in both python3.6 and python3.8. Thank you!

2

Hi @jonahrosenblum,

We’re really sorry you ran into this. I think the problem might be that you’re looking at the documentation for the master branch (the nightly builds), but you’re running the latest pip release (Ray 1.2.0). There was a recent breaking API change in the nightly builds.

Here’s the documentation for Ray 1.2.0: Ray Serve: Scalable and Programmable Serving — Ray v1.2.0
The sample code there should run properly. Alternatively, you can install the nightly builds using these instructions: Installing Ray — Ray v2.0.0.dev0

To check your Ray version, you can run ray --version in the terminal, or check ray.__version__ from Python.

1 Like
3

Ah, this makes a lot more sense. I appreciate you clearing things up for me!

4

Hi @architkulkarni

I read your blog post here How to Scale Up Your FastAPI Application Using Ray Serve | by Archit Kulkarni | Distributed Computing with Ray | Medium.

This is exactly what I want to implement, but my version of ray is 1.9. Unfortunately the link you shared previously to the installation documents no longer works.

I’m getting the same error as above, AttributeError: module ‘ray.serve’ has no attribute ‘BackendConfig’

5

@architkulkarni

Update: I found the latest installation directions here Installing Ray — Ray v1.9.2.
And upgraded to ray, version 2.0.0.dev0.

However, I am still getting the following errors: AttributeError: ‘Client’ object has no attribute ‘create_backend’

6

Hi Mark, thanks for trying out serve ! So the short answer is “backend” was an old term in serve that we deprecated recently to “deployment”, but our old posts might still be using it.

For the simple example:

import ray
from ray import serve

# This will connect to the running Ray cluster.
ray.init(address="auto", namespace="serve")

@serve.deployment
def my_func(request):
  return "hello"

my_func.deploy()

You can find more examples and tutorials in End-to-End Tutorial — Ray 2.0.0.dev0

7

+1 to what Jiao said. Since that blog post was published, we’ve improved our FastAPI integration, check it out here! Calling Deployments via HTTP and Python — Ray v1.9.2