Memory leak in ray serve 2.1.0

How severe does this issue affect your experience of using Ray?

  • High: It blocks me to complete my task.

I use ray[serve] to deploy my web serve. I found that there are memory leak happened.

I use the single node with the steps:

  1. ray start --head
  2. python my_serve.py

the key codes is as following:


my_serve.py
ray.init(address=‘auto’, namespace=‘serve’)
serve.start(http_options={‘host’:‘0.0.0.0’, ‘port’:9099}, detached=True)
MyServe.deploy()

@serve.deployment(name=…, num_replicas=4, ray_actor_options={‘num_gpus’: 0.1}, route_prefix=‘/detect’)
class MyServe:
def init(self):

@serve.batch(max_batch_size=16, batch_wait_timeout_s=10)
async def handle_batch(self, flask_requests):
	for flask_request in flask_requests:
		info = await flask_request.json()
		...
	
	return response

async def __call__(self, request):
	return await self.handle_batch(request)

Could anyone give me some advise?

We had a recent fix for memory leak in [Core] Close GCS channel after health check to fix one memory leak in `dashboard.py` by architkulkarni · Pull Request #30346 · ray-project/ray · GitHub. Can you please try out the nightly release to see whether the leak still exist? Installing Ray — Ray 3.0.0.dev0