Hello guys, as the following sample code shows, I make my servable methods
async def for the purpose of processing HTTP requests concurrently.
The expected output is “one one one one one two two two two two", however, the actual output is “one two one two one two one two one two” .
It seems that the requests are not executed asynchronously but serially. Where is my mistake？
from random import random import requests import ray from ray import serve import time import asyncio @serve.deployment(num_replicas=10, route_prefix="/composed") class ComposedModel: def __init__(self): print("hello, world!") # This method can be called concurrently! async def __call__(self, starlette_request): print("One") await asyncio.sleep(5) print("Two") return 0 ray.init() serve.start() ComposedModel.deploy() for _ in range(5): resp = requests.get("http://127.0.0.1:8000/composed", data="hey!")