Is it possible to send a request to a single node in a DAG?

Ray Libraries (Data, Train, Tune, Serve)
1

Let’s say I have a DAG composed of three nodes, similar to this example: ray_dag.py

I can run that from the command line using “serve run ray_dag:graph” and send it queries via http using this code: client.py

That works great. But, now let’s say that I wanted to ONLY ping a single node in the graph (e.g., func1 from the ray_dag.py example above). I know I can create a second instance of it and serve it separately, but can I ping this instance that was already instantiated within the DAG?

The reason I ask is because I’m planning on creating a DAG composed of several ML models (one model per node) for inference. In 99% of cases, I’ll need to run the data through all the nodes. But there will be a few cases where I only need to run the data past a single node and I don’t want to have to create a separate instance of the model (because it’s pretty large).

How can I achieve this? Is it possible to define some sort of routine that only pings the individual node?

Thanks!

2

Absolutely!

You can change the last line of ray_dag.py into

graph = DAGDriver.bind({"/inference": func3_output, "/func1": func1.bind(unpack_output)})

This will create the routing /inference binding to func3_output and /func1 binding to just func1 (and unpack_output)

Some examples can be found: 1.x to 2.x API Migration Guide — Ray 2.5.1

3

Oh that’s exactly what I need, thank you!!