How severe does this issue affect your experience of using Ray?
- High: It blocks me to complete my task.
I use ray[serve] to deploy my web serve. I found that there are memory leak happened.
I use the single node with the steps:
- ray start --head
- python my_serve.py
the key codes is as following:
my_serve.py
ray.init(address=‘auto’, namespace=‘serve’)
serve.start(http_options={‘host’:‘0.0.0.0’, ‘port’:9099}, detached=True)
MyServe.deploy()
@serve.deployment(name=…, num_replicas=4, ray_actor_options={‘num_gpus’: 0.1}, route_prefix=‘/detect’)
class MyServe:
def init(self):
…
@serve.batch(max_batch_size=16, batch_wait_timeout_s=10)
async def handle_batch(self, flask_requests):
for flask_request in flask_requests:
info = await flask_request.json()
...
return response
async def __call__(self, request):
return await self.handle_batch(request)
Could anyone give me some advise?