Understanding @serve.deployment

Ray Libraries (Data, Train, Tune, Serve) Ray Serve
1

Dear all,

I am trying to understand how Ray Serve works. I did a simple Python script and used the corresponding decorator (@serve.deployment) to apply Ray Serve philosophy. However, when executing the script, an “Attribute” error appears.

------------------------------------------------------------ SCRIPT ------------------------------------------------------------

father_son.py script

Import libraries

import random
import ray
import time
import asyncio

from ray import serve

Initialize Ray

ray.init(ignore_reinit_error=True)
serve.start(detached=True)

Father class that generates a vector of two random numbers

@serve.deployment
class Father:
def generate_numbers(self):
“”“Generate two random numbers between 0 and 9.”“”
num1 = random.randint(0, 9)
num2 = random.randint(0, 9)
return [num1, num2]

Son class that performs the multiplication of two numbers

@serve.deployment
class Son:
def multiply_numbers(self, numbers):
“”“Multiply two numbers from a given vector of two elements.”“”
if len(numbers) != 2:
raise ValueError(“Input must be a vector of two elements.”)
return numbers[0] * numbers[1]

def main():
# Deploy Father and Son using serve.run with deployment options
Father.deploy()
Son.deploy()

# Get handles to deployed services
father = Father.get_handle()
son = Son.get_handle()

# Function to perform workflow
async def do_workflow():
    numbers = await father.generate_numbers.remote()
    result = await son.multiply_numbers.remote(numbers)
    print(f"Numbers: {numbers}, Product: {result}")

# Run the workflows in parallel
(do_workflow() for _ in range(10))

if name == “main”:
start_time = time.time()

# Run the main function
main()

end_time = time.time()
time_taken = end_time - start_time
print(f"Time taken to execute the code in parallel: {time_taken}")

ray.shutdown()

------------------------------------------------------------ /SCRIPT ------------------------------------------------------------

---------------------------------------- CMD OUTPUT ----------------------------------------
odriovik@odriovik:~/Escritorio/Python$ python3 father_son_ray_serve.py
2024-08-29 14:47:15,423 INFO worker.py:1772 – Started a local Ray instance. View the dashboard at http://127.0.0.1:8265
(ProxyActor pid=11615) INFO 2024-08-29 14:47:18,240 proxy 10.0.2.15 proxy.py:1179 - Proxy starting on node d4dbd04f883aae16a82d7a8aa2639eee76da576121ad952c426a3d16 (HTTP port: 8000).
2024-08-29 14:47:18,356 WARNING api.py:346 – The default value for max_ongoing_requests has changed from 100 to 5 in Ray 2.32.0.
2024-08-29 14:47:18,356 WARNING api.py:397 – The default value for max_ongoing_requests has changed from 100 to 5 in Ray 2.32.0.
2024-08-29 14:47:18,366 WARNING api.py:346 – The default value for max_ongoing_requests has changed from 100 to 5 in Ray 2.32.0.
2024-08-29 14:47:18,367 WARNING api.py:397 – The default value for max_ongoing_requests has changed from 100 to 5 in Ray 2.32.0.
Traceback (most recent call last):
File “/home/odriovik/Escritorio/Python/father_son_ray_serve.py”, line 55, in
main()
File “/home/odriovik/Escritorio/Python/father_son_ray_serve.py”, line 35, in main
Father.deploy()
^^^^^^^^^^^^^
AttributeError: ‘Deployment’ object has no attribute ‘deploy’. Did you mean: ‘_deploy’?
---------------------------------------- /CMD OUTPUT ----------------------------------------

I would appreciate it if someone helps me with this issue. Big thank you to Ray Community.

2

To deploy your Serve application, you would first need to call bind() to create the application object. Then you can call serve.run to deploy them. Basically instead of calling those .deploy() and .get_handle() you can do the following

# create Serve applications
father_app = Father.bind()
son_app = Son.bind()

# Deploy the application and get the handles
father_handle = serve.run(father_app)
son_handle = serve.run(son_app)

Then you can do your workflow and call those remote functions.

Also note you can pass the deployment handle in the constructors if you are trying to do model compositions. More details on this doc Deploy Compositions of Models — Ray 2.35.0