Regarding HttpProxyActor

Ray Libraries (Data, Train, Tune, Serve) Ray Serve
1

I am trying Ray Serve, I have written a sample deployment , to simulate real time environment I have used time.sleep(0.5) in my method.

Now when I call the serve deployment endpoints concurrently for 1000 invocations HttpProxyActor Cpu usage is going to 100% and HttpProxyActor is getting stopped.

What parameters would help me over here? Please reply …

def summarize(text, endpointname):
# Load model
reversedString = reverseMyText(text)
time.sleep(0.5)
return reversedString

2

Hi @Vikas_Mavoori , could you give more details about the HTTPProxyActor getting stopped? How does it get stopped, is there an error or traceback you can share?

3

Hi Archit,

I found the solution to my issue. I have used time.sleep which is stopping the execution synchronously.

Later to resolve it I used asyncio and it solved the issue.

Anyways, thanks for the reply.

Regards
Vikas

1 Like