Custom replica metadata, access to custom property

Ray Core
1

How severe does this issue affect your experience of using Ray?

  • Medium: It contributes to significant difficulty to complete my task, but I can work around it.

Hi,
I’m using the gRPC direct ingress feature of Ray to deploy ML models with a direct gRPC connection between Client and Server. I have a custom “registry” that manages models/IP/ports etc. So when a model replica is initiated it starts a gRPC Server with a individual port for that actor. The port is a custom Class variable which I would like to get somehow from outside of the Actor. So ideally, I could get the port number for replica XY with the actor_id…Is there way to access properties of a replica or to somehow add custom info to an actors metadata? So my registry deployment can check which actors are active and which port they have on a regular basis.
I’d appreciate any help.
Thanks!

2

Hi @bananajoe182 what if you just add a method in the actor handler and return the port from there? Later if you want to know which actor is using which port, you can just call that? You can also add other meta infos there too, like replica id or name. Ray itself doesn’t have this support.

3

Hi yic,
thanks for the answer!
I’m deploying the models like this from a jupyter notebook (for testing). Could I add a method in that class or would I need to deploy the model differently?:

@serve.deployment(num_replicas=2, ray_actor_options={"num_cpus": 4, "num_gpus":0.5, "runtime_env": runtime_env})
class Model(rayNuke_pb2_grpc.PredictAPIsServiceServicer, CustomgRPCIngress):
    def __init__(self):
4

How would I add a method in the actor handler when I deploy my Classes using:
@serve.deployment
Class …

app_class = Class.bind()
and then using serve deploy config.yaml